{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZXV1EHipyGUE"
   },
   "source": [
    "# 图像分类\n",
    "\n",
    "在此项目中，你将对 [CIFAR-10 数据集](https://www.cs.toronto.edu/~kriz/cifar.html) 中的图片进行分类。该数据集包含飞机、猫狗和其他物体。你需要预处理这些图片，然后用所有样本训练一个卷积神经网络。图片需要标准化（normalized），标签需要采用 one-hot 编码。你需要应用所学的知识构建卷积的、最大池化（max pooling）、丢弃（dropout）和完全连接（fully connected）的层。最后，你需要在样本图片上看到神经网络的预测结果。\n",
    "\n",
    "\n",
    "## 获取数据\n",
    "\n",
    "请运行以下单元，以下载 [CIFAR-10 数据集（Python版）](https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz)。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 35,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 655,
     "status": "ok",
     "timestamp": 1518436306157,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "r7t4zy0fyGUF",
    "outputId": "1337a4a9-5d36-4382-cc1f-2f96c0159f59"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "All files found!\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "from urllib.request import urlretrieve\n",
    "from os.path import isfile, isdir\n",
    "from tqdm import tqdm\n",
    "import problem_unittests as tests\n",
    "import tarfile\n",
    "\n",
    "cifar10_dataset_folder_path = 'cifar-10-batches-py'\n",
    "\n",
    "# Use Floyd's cifar-10 dataset if present\n",
    "floyd_cifar10_location = '/input/cifar-10/python.tar.gz'\n",
    "if isfile(floyd_cifar10_location):\n",
    "    tar_gz_path = floyd_cifar10_location\n",
    "else:\n",
    "    tar_gz_path = 'cifar-10-python.tar.gz'\n",
    "\n",
    "class DLProgress(tqdm):\n",
    "    last_block = 0\n",
    "\n",
    "    def hook(self, block_num=1, block_size=1, total_size=None):\n",
    "        self.total = total_size\n",
    "        self.update((block_num - self.last_block) * block_size)\n",
    "        self.last_block = block_num\n",
    "\n",
    "if not isfile(tar_gz_path):\n",
    "    with DLProgress(unit='B', unit_scale=True, miniters=1, desc='CIFAR-10 Dataset') as pbar:\n",
    "        urlretrieve(\n",
    "            'https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz',\n",
    "            tar_gz_path,\n",
    "            pbar.hook)\n",
    "\n",
    "if not isdir(cifar10_dataset_folder_path):\n",
    "    with tarfile.open(tar_gz_path) as tar:\n",
    "        tar.extractall()\n",
    "        tar.close()\n",
    "\n",
    "\n",
    "tests.test_folder_path(cifar10_dataset_folder_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FYI24o-jyGUM"
   },
   "source": [
    "## 探索数据\n",
    "\n",
    "该数据集分成了几部分／批次（batches），以免你的机器在计算时内存不足。CIFAR-10 数据集包含 5 个部分，名称分别为 `data_batch_1`、`data_batch_2`，以此类推。每个部分都包含以下某个类别的标签和图片：\n",
    "\n",
    "* 飞机\n",
    "* 汽车\n",
    "* 鸟类\n",
    "* 猫\n",
    "* 鹿\n",
    "* 狗\n",
    "* 青蛙\n",
    "* 马\n",
    "* 船只\n",
    "* 卡车\n",
    "\n",
    "了解数据集也是对数据进行预测的必经步骤。你可以通过更改 `batch_id` 和 `sample_id` 探索下面的代码单元。`batch_id` 是数据集一个部分的 ID（1 到 5）。`sample_id` 是该部分中图片和标签对（label pair）的 ID。\n",
    "\n",
    "问问你自己：“可能的标签有哪些？”、“图片数据的值范围是多少？”、“标签是按顺序排列，还是随机排列的？”。思考类似的问题，有助于你预处理数据，并使预测结果更准确。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 451,
     "output_extras": [
      {
       "item_id": 1
      },
      {
       "item_id": 2
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 624,
     "status": "ok",
     "timestamp": 1518436307004,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "YuYLhWnGyGUN",
    "outputId": "4cd91037-e177-4ab0-9b07-64a6273598fc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Stats of batch 1:\n",
      "Samples: 10000\n",
      "Label Counts: {0: 1005, 1: 974, 2: 1032, 3: 1016, 4: 999, 5: 937, 6: 1030, 7: 1001, 8: 1025, 9: 981}\n",
      "First 20 Labels: [6, 9, 9, 4, 1, 1, 2, 7, 8, 3, 4, 7, 7, 2, 9, 9, 9, 3, 2, 6]\n",
      "\n",
      "Example of Image 6:\n",
      "Image - Min Value: 7 Max Value: 249\n",
      "Image - Shape: (32, 32, 3)\n",
      "Label - Label Id: 2 Name: bird\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfMAAAHxCAYAAAB5x1VAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAHPdJREFUeJzt3cmO5PeVHeBfRGRE5DxVZc2kWKIo\niaQmtFpwe9GwbMDdC6+987P4aey1d1423IYtQGgZ3dZIUhQp1jzkUDlGxuxnuNcG2jfxffuDmxnx\nzzgVmzqd5XLZAIC6uv/cPwAA8H9HmQNAccocAIpT5gBQnDIHgOKUOQAUp8wBoDhlDgDFKXMAKE6Z\nA0BxyhwAilPmAFCcMgeA4pQ5ABS38s/9A/y/9J//8d+n9lz/59+9Dme2Vr+fOdU21rfDmX4n9zZt\nbvRTuds7D8KZvfVHqVu7OzvhzMvDJ6lbX73936nc9sOLcObWw8vUrf7wKpwZXb5L3VpdHYQzvc5u\n6tZiPkvl5vPzcGZvO/csDofr4cxKi/98rbV2ejZO5Y5exz8Lri/if2OttXY13gxnli03qX1y/DKV\nu7qKv45nF6epW8sWf4ZPjuOfHa219p/+4y860Yxv5gBQnDIHgOKUOQAUp8wBoDhlDgDFKXMAKE6Z\nA0BxyhwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMXdqNW03jCX27gdX7b59f/6RerWe/f+IpzZ2lhL\n3bqe9FK50Xl8+Wi0Gx75aa21NuvEV8L2HuQe24/ey+VGq/FVvfNFbslscRZfMhvON1K3lsP4+zyd\nx9+v1lpb6cUXuFprbX/7djizPsgtd00vt8KZs8v7qVvnR2ep3JMvvglnesNF6lbrT8ORZ89fpU5t\nbcaf+9ZauzifhzOzWe5WSyzCLZIvfYZv5gBQnDIHgOKUOQAUp8wBoDhlDgDFKXMAKE6ZA0BxyhwA\nilPmAFCcMgeA4pQ5ABSnzAGguBs1tPL8zVEq9+DxXjjT68VHGVprbX/z24lUfPCgtdaef/1VKvf1\n85fhzMMHuQGOy2X8ddxbOUndmm1/lsp1N+PP1XjaT906fzcLZ/ZX1lO3BolBku2d3GDK1tqjVG48\njT/7k1luxKTN4qsYp68PUqdOvsp99H7xq38KZzbeiz9TrbX28Dt3wpnVjdxzf3aee8/G14nfrZP7\nGQ+P3oYzk+l16laGb+YAUJwyB4DilDkAFKfMAaA4ZQ4AxSlzAChOmQNAccocAIpT5gBQnDIHgOKU\nOQAUp8wBoDhlDgDF3ajVtC++OE/lPvh2fPno8ffeT9366o9fhjOXVxepWxtbuTWt89FpOPPbz3+T\nurX54KNw5tbWJHVr1o2vYrXW2rOvEmt8y9xrvzd4ED/VcqtYq4P4c7+/czd16+J0kMp99of477a3\ncS91a2s7/t1mequXunX5PPczvnq9G848fpT7Gdc346/HbJF77ifXuc+4lUH8Zzw5zvXE1WV8Aa2T\ne+lTfDMHgOKUOQAUp8wBoDhlDgDFKXMAKE6ZA0BxyhwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMXd\nqKGVp0/mqdyyjcKZs1tPU7cm3fiIyXxlmrq1u7efyn30vcfhzOs38d+rtdYup/Hxgl//LjF80lqb\ndXPPx+7t+BhMW+bGHPrD+Ouxt597nzfXb4cz52ed1K3D1+NUbjGJf0Stbm+lbp1N9sKZ31x/O3Vr\nvH8rleve+SacWV/N/b2cvDsOZ16+yD33s3FuLGg6jv+9XFyepW7NZvGfcXUwTN3K8M0cAIpT5gBQ\nnDIHgOKUOQAUp8wBoDhlDgDFKXMAKE6ZA0BxyhwAilPmAFCcMgeA4pQ5ABSnzAGguBu1mjYb91O5\nd28m4cz06iR1a7ixDGf27uVWsZbD3BLRne9shjNni4vUrYtR/LVfa7nX4+govrDUWmtbg51w5sGj\n3dStaXsTzpwucr/X5fFhOLPai78WrbV2ER8mbK21trUdX52aDXJ/m28u74Qz//W/xJ/f1lpbLF+k\nch8O4j9jb9lL3Tp8EV8Xm1zHP99aa623klvju57GFyWXndytza34s99Z5m5l+GYOAMUpcwAoTpkD\nQHHKHACKU+YAUJwyB4DilDkAFKfMAaA4ZQ4AxSlzAChOmQNAccocAIpT5gBQ3I1aTRt2cqtp01F8\ndWrv3r3UreevX4czZ9fPU7eW3S9SuR//4LvhzL/829zrsTHYCmemV/FMa6198UVuuuvs5G04s7YW\nX/tqrbX5YB7OPDt7krp1ayu+OPVgb5C6tbW/lsoNEt83Lme55a4/PfsmnPnqf5ymbk3O/5TKdd6L\n37t6E18/a621+99aD2fWdnPPR+vmlv+6vfi99fVcT0wSC4/9bvw1zPLNHACKU+YAUJwyB4DilDkA\nFKfMAaA4ZQ4AxSlzAChOmQNAccocAIpT5gBQnDIHgOKUOQAUd6OGVs5PLlK57dvxYYajs5epW6ub\nnXDm4nKWujWdxUc7Wmvts99/Hc68fJ4b+9jaWg1n7t59L3Xrzge5EYirby7Dmadvc0Maa1uLcObW\nwXbq1t52fNyi232WurUyiL/PrbU26O6EM7PJ7dStxTT+t9kWJ6lbH/8wN9Dy/cfx3Nb6OHVr7yD+\nLF5dbaRuTSa5v83zo/hw1XwS/71aa21tkBhNmedGfzJ8MweA4pQ5ABSnzAGgOGUOAMUpcwAoTpkD\nQHHKHACKU+YAUJwyB4DilDkAFKfMAaA4ZQ4AxSlzACjuRq2mdRaJ1aPWWnclsWQ2epe6dffunXCm\n1+LLUa219uLFNJU7W8YXrs5OJqlbK6tvw5mjy3imtdZ2tvZSudXNtXBm+9aj1K21YfxP8u7e/eSt\nXiKVe6am09yC33R6FM4s+7nvKGcnB+HMdm6wrv38395K5YbtTThz/95m6tYg8Xx88ZvcItnxyVUq\nd302CmeWyTXJndvx13GevJXhmzkAFKfMAaA4ZQ4AxSlzAChOmQNAccocAIpT5gBQnDIHgOKUOQAU\np8wBoDhlDgDFKXMAKO5GDa1cnJ+ncr3L+L9ptvq5l256FR8U6LbcCMHacJzKdTvxoZWtvd3UrXlv\nFs6MJrmhlavXuTGYxw8/DWd21uKjHa211qbLeOQ0N6Sxt7EeD/Vzr+HV9WUq11biz8eil/vb/OrL\nfjizd3eYuvUXP80Nray1j8KZ6fwidev6Mj5ANZu+Tt2ajHKf3cNe/PVf28i9Z73ELlGnmxueyfDN\nHACKU+YAUJwyB4DilDkAFKfMAaA4ZQ4AxSlzAChOmQNAccocAIpT5gBQnDIHgOKUOQAUp8wBoLgb\ntZrWG+b+bTK6noYzF9/kVn7Gh6Nw5s6D+JJWa61trOXWgU5H78KZrZXcQtv+3fgU0du3ydWjeW5d\nbD6O/4zXF7mlu2FnI5zp9nKLdceH8Z9xZWOeunV0nns+RheJxa+V3Ovx9Hn84/D+o9PUrdXNs1Ru\n5Tq+WjcaJdbxWmvLcfx1fPQwt6q3k1nwa629+ia+xrexmXw9uvHfrRMf4kvzzRwAilPmAFCcMgeA\n4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKU+YAUJwyB4DilDkAFKfMAaC4G7Wa1lnOUrnldXwJ\n6mD7dupWbxT/GWfnuemdxTD39k6u44twh4fx9aLWWlv2O+HMRj++LNZaawd3HqRyd27F3+uD3Tup\nW20aX2jr9wbJU/FFsrPLt6lbz15/ncq9evY6nDmOR1prrc3GPwpntnZzr8erw9+ncjud+OLX+uCT\n1K07D74bzjx4uJW61ZmtpnLnH6+FM5NZYomvtTbvxFcGr8bxlcws38wBoDhlDgDFKXMAKE6ZA0Bx\nyhwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHE3amilTa9TscFKfMhkczBM3erP4y/5\nbBIfgmmttc4w93qsr8Z/t6M309SteeJH/Pjb76VuPbz1OJVbWYkPmVxf5sZx+i0+HNHpxcdqWmvt\nYrIMZz7/+knq1st3uVx3Gn/2F+9yr/3+Mj6K8d293Peh2VXub3OyEh8k6U0PU7c63fjvNljL/V53\nb3+Uyt3efj+cObs8Sd0aT8fhzMbKrdStDN/MAaA4ZQ4AxSlzAChOmQNAccocAIpT5gBQnDIHgOKU\nOQAUp8wBoDhlDgDFKXMAKE6ZA0BxyhwAirtRq2nbO+up3OpGfKlquZJbqtrY3QxnZvP4Wk9rrc1m\nl6ncxelVONO7iC9wtdbacCX+2rdRbhWrjW6nYp2Vg3BmPou/z621NuzHc9N5brHuNDEetTz7OHVr\nbbqfyy3j7/Ww9zB169W7X4UzH6zcSd16tPqDVG7ajb/Xo6uL1K3TyctwZnF8mrrVWZylcrsb8dyi\nm1u8PD+LL/gNNvZStzJ8MweA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKU+YAUJwyB4DilDkA\nFKfMAaA4ZQ4Axd2ooZXeODf2Me/MwpnpMjducZX4Ea8ucoMp/UHu9djuxAdrht1e6tZgth3ObPS+\nlbrVG3+Yyi1Gd8OZtf5u6labx/993ZnHByBaa+3+Vvx1vLf7V6lbo/l5Knd5PApnvn7zTerW3srv\nwpmdZW7c6f07uWfxD6/+FM50O7mxj34n/hk3GeeexetRLjfa/GU4Mx8kxp1aa2fXq+HM+bv4WE1r\nrbUf/rtwxDdzAChOmQNAccocAIpT5gBQnDIHgOKUOQAUp8wBoDhlDgDFKXMAKE6ZA0BxyhwAilPm\nAFCcMgeA4m7UatriTW4lbLG2CGcm3evUrcHaIJ7p30rd6k7iv1drrS1nk3BmMcs9Snce/CSc6c+/\nl7r19kVuLam/Ev/dZmvxJb7WWptPxuHMaBR/v1prbXUtvgLVTX5i7OzeT+UG2/E1vuOD3HM/2Igv\noJ1dn6RuvR79NpXbvBf//rU6z62mja83w5ne/EHq1rJ1UrlXx/8Yzgz7W6lb+/s/Cme60/hrmOWb\nOQAUp8wBoDhlDgDFKXMAKE6ZA0BxyhwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAo7kYNrXzy\n6Kep3Hx9GM/0+6lb93dvhzOrO9upW51Fbrzg7dsn4czxZW5YpLf6nXDm+no3dWs0zY3jrK6dhjOT\nSe7W6PIqnLm8vEzdms/niUzufd7eyo1brG3Gx3Gevz1O3bruxYdWXl6+Td3aPMqNQvX24q/H9OzP\nqVvr3fgo1N7aB6lbK4PcZ9VsHP8ZN4a54apH9z4KZ/rtYepWhm/mAFCcMgeA4pQ5ABSnzAGgOGUO\nAMUpcwAoTpkDQHHKHACKU+YAUJwyB4DilDkAFKfMAaA4ZQ4Axd2o1bQf/fjnqVx3J77o1N3cSN3a\nXY0vM/WG8VW31lrrtdyy2+8+/1U4c/TkderW16/iK2H9ldwi2dpmL5UbTM/DmeU0vubUWmuXp6Nw\nZrYcp24NBvHn4+oi/lq01tpXf/5TKre5Gn8d54vcx9rFdBLOvD0/St36cPpBKnf8fBrOPPnzH1K3\n+pP438vuZu5z4MEHO6nc6Sy+kLfYjX8Gt9bafj++kLc5zK0FZvhmDgDFKXMAKE6ZA0BxyhwAilPm\nAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKU+YAUNyNWk37zo9+lsot+6vhzHwlvl7U\nWmsrvctwpjeP/3yttdZZy62EXf12Hs48f5pbjzq+jue2NjdTt2avcu/Z+jB+787+ndStW9vx9aiL\nq/gz1Vprk0l8fW56HV8Wa621i3dnqdz1YhbOdBfJn/H6aTyT+Plaa+1skVuf63SX4Uy/czd16/df\nxpfudm7nfq+Tldy6WH8j/jd9kVhBbK21o5OLcObx3b9M3frp3f8QzvhmDgDFKXMAKE6ZA0BxyhwA\nilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKu1FDK+s78ZGK1lqbLeL/ppl3Uqda\n68eHGRbLq9Sp1c3c0Mr08m048/qPv0/dWm5uhDMH9z5N3fry8xep3KizFs50LsepWysP40ManRbP\ntNbayyd/Dmcur3KDKVdX8ZGK1lrrzeOjP51lbnimrb4LR5b9furU01fxUZfWWtvbif+9vPf+o9St\n8Tj+3I8mufd5Ms7ltvbjr//1eJG6NTk7DWeGLT5W01pr7QfxiG/mAFCcMgeA4pQ5ABSnzAGgOGUO\nAMUpcwAoTpkDQHHKHACKU+YAUJwyB4DilDkAFKfMAaA4ZQ4Axd2o1bRubiSsLefxJbPpdJK6NZtf\nhzOLQW6Ba3E+TeU6F0fhzOziderW3sHjcGb8Nnfr8k1uqWq2iE/kTS9y62JHid+tN8w9+KPReSKT\n+73Or+LPVGut9bqJj6he/G+stdYePY7funN/O3VrfZiKteUyvpB3OX2VuvX4g/fDmZX5w9Stq8nv\nUrnuyrNwZjKPr8G11trGZnx9bpH7CE7xzRwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkD\nQHHKHACKU+YAUJwyB4DilDkAFHejhlZGk9zAwmQ0D2euJ6PUrfkynpvNjlO3Zi03BnN1Gh/g6A7j\nYySttbayEX8E3x3mxj4OX8ZHGVprbbKMP1ez+VXq1ubu/fit69zQymIS/xmvRm9Tt67nb1K5zqAf\nzqz042MkrbV2+1H8tf/Od+NDQa219uooNxY0SOy6dLq5W5PL+OfOvb0fpm617oNUbLkZ/yz4/LOT\n1K37B3fDmY3heupWhm/mAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKU+YAUJwyB4Di\nlDkAFKfMAaA4ZQ4Axd2o1bT5IrfctUiMLK0OtlK3puPLcGby7mXq1vH0XSq3fms3nPlXf/PXqVsv\nruILRk+Pn6duHXw4TOUWnfi/eefT3GrapF2EMxvbucWpN0/jz9X1JLea9tFP9lO5thb/4zw6PUqd\n2r2zFg914qturbU2ush9Vu0fbIQzs2VuJez23Z1w5uAg9/2w272dyr0bxVfJDnZzP+OwF7/15kVu\nXTPDN3MAKE6ZA0BxyhwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKU+YAUJwy\nB4DibtRq2mSySOU6iZehs0j+O2gev9Vfza19re7mlt02L+O586+epm795acH4cyHn/ZSt1r3bio2\nGcXf63/477nX4/AwvsK1tpV7n69G8YW2nf3cStiPfvatVO7rN5/HQ1u5RbIH798LZ/b27qdubW7k\nlu5Gs9fhzPnVOHVrsYy/188Of5u6tb+bW00bX8WX3XbW9lK3pqN5ODO+zr32Gb6ZA0BxyhwAilPm\nAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKU+YAUJwyB4DibtTQynwS/4/wW2ttfn0d\nzqysLFO3OiujcGZrey11az56l8o9f/KHcOaPv/0ydWtr9fvhzPX+q9St0XSSyt1aez+c6S7iz1Rr\nrR3sfTecGa5tpG6Np/Fhop3bu6lb01nutT8/PwxnHj6Kj/e01lpnHn/P/v7vfpm61V/PjULdeT/+\nGTfo5YaaXr14G85M5kepW8cXueGZ/dWH4czO5nbq1mwl/t13tsi9zxm+mQNAccocAIpT5gBQnDIH\ngOKUOQAUp8wBoDhlDgDFKXMAKE6ZA0BxyhwAilPmAFCcMgeA4pQ5ABR3o1bT+v1pKje9uApnVga9\n1K3reXwF6sXrX6duffar36RyW73NcGZjupq69Yf/9k/hzPCDTurWUWIdr7XW1j+ML4V98Gg9devZ\n63E4M5/MUrdWBoNw5m5itau11hbLi1zuKv4zrndzK2Fff/7HcOYXv3yWuvXok9xH72Ir/v2rP7uV\nujU7i7/2+we53+vPX/8plfvs9Dic+Zt//depW/cexdcrL2e5FbkM38wBoDhlDgDFKXMAKE6ZA0Bx\nyhwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHE3amjlZPo0lZuMR+HMZXybpbXW2ut3\n8fGTFyd/n7p1+OpdKnev/2k4c6uTG545G8V/xv6r7dStwSg3SPJs/kU4871/863UraNF/PU4eZH7\nMz64Hx9N+dHPcv/+X93IDfEcHr4fzrx9Gx/faK21jc2tcObjjx+lbm0/yn2ALOfxz6r5NPd8vHp+\nGc5cHuduTca5EaR3F6fhzPOPb6dubWzdCWdeHuZGsjJ8MweA4pQ5ABSnzAGgOGUOAMUpcwAoTpkD\nQHHKHACKU+YAUJwyB4DilDkAFKfMAaA4ZQ4AxSlzACjuZq2mXbxM5S7PXoUz81F8Uai11t5d/Cmc\nWVzHl5Jaa21nfZnKXZ1+Gc5s7OdW07qb8QW0/upm6tb2dCeV695dD2f2DnIrYds7nXDmyee5dbxO\ni79nx69z//4fzw5Tubv34qtkT5/nFsmODuN/08v+JHXrTu7xaMNh/PnodOKZ1lobjxfhzMsvzlK3\nNvq5F+S7P3kczlwkltZaa+3wJP552h/GlwmzfDMHgOKUOQAUp8wBoDhlDgDFKXMAKE6ZA0BxyhwA\nilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAo7katpo3O4+tnrbXW6b0NZ/pb16lbO+vxBaPxV/HV\nrtZa2zqYpnLT28fhTKe/n7r1YP8H4cyz57n3+fSPubWkTx5+Es5sbuYW6957FF/hOnoRf79aa+2r\n38d/xtFZbh2vt55bMhusxRcD7z7IPYuvnsWX3caL3HpiW+aej06LL5lt7w5Ttx5/uBfOvP3yaerW\nbJpbTTs7Hoczr17mlt3G8/g64a3bu6lbGb6ZA0BxyhwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUp\ncwAoTpkDQHHKHACKU+YAUJwyB4DibtbQyvFnqVxvGP/P+sed+OBBa60NtuKDAvc/fZC6NZ3OU7nZ\nMP5vvMXpdurW2Zv4AMfFu9xox+hlfLSjtdZ+8w9fhDO3tnN/Wt3+ZjjzVz/PDfF88PhuOLN/EP9b\naa217Tu5sY+1W/G/l273XurW4fPH4cyb4y9TtxbDJ6lcm/YTxwapU4P1eK6Te5vb1mbu83SxOA9n\nLi5mqVuzbjy3urqWupXhmzkAFKfMAaA4ZQ4AxSlzAChOmQNAccocAIpT5gBQnDIHgOKUOQAUp8wB\noDhlDgDFKXMAKE6ZA0BxN2o17d5a7te5GnbCmZUWX3NqrbXlSvzfT4O93NrX5GQrlbt6E8+c/OEo\ndWtwEV8J2x7fSt2a9XP/dh0vJ+HMYp5bMjt5fR3OnE/jP19rrX378e1wZjzNLU4dP809H92L+MO4\nupl7nx8//nE4c/dhbhXr5Do3L/b2bXwlbDHJfVb1BvHPxR//iw9yt+YnqdyixRcUR7Pc52kn8Znf\n6S5TtzJ8MweA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKU+YAUJwyB4DilDkAFKfMAaA4ZQ4A\nxd2ooZXbs71Ubnx/O5x58+xd6tabZ6/Dmdn6OHVrZbKTynWfz8OZ1ePcAEfrJgYnZvH3q7XWNr6T\nGz+59WF8LKGXfO3bm/hz9eqr+DPVWmvzk/hIxZ3HyWdq0Uvl1sb3w5nj08vUrf78SThz6+7d1K17\n+5+kcvPr5+HM0+e552NtM/73sneQG5CZXefGYFb68TGYdpgbPxmfxj8Xp9fJz8UE38wBoDhlDgDF\nKXMAKE6ZA0BxyhwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACK6yyXuQUZAOD/\nD76ZA0BxyhwAilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKU+YAUJwyB4DilDkA\nFKfMAaA4ZQ4AxSlzAChOmQNAccocAIpT5gBQnDIHgOKUOQAUp8wBoDhlDgDFKXMAKE6ZA0BxyhwA\nilPmAFCcMgeA4pQ5ABSnzAGgOGUOAMUpcwAoTpkDQHHKHACKU+YAUJwyB4DilDkAFKfMAaA4ZQ4A\nxSlzAChOmQNAccocAIpT5gBQnDIHgOKUOQAUp8wBoLj/A1AoH6PQ87rUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5fc7d0a780>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 248,
       "width": 249
      },
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "\n",
    "import helper\n",
    "import numpy as np\n",
    "\n",
    "# Explore the dataset\n",
    "batch_id = 1\n",
    "sample_id = 6\n",
    "helper.display_stats(cifar10_dataset_folder_path, batch_id, sample_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YiVSeJQRyGUR"
   },
   "source": [
    "## 实现预处理函数\n",
    "\n",
    "### 标准化\n",
    "\n",
    "在下面的单元中，实现 `normalize` 函数，传入图片数据 `x`，并返回标准化 Numpy 数组。值应该在 0 到 1 的范围内（含 0 和 1）。返回对象应该和 `x` 的形状一样。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 35,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 655,
     "status": "ok",
     "timestamp": 1518436307997,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "L8B56Q5cyGUR",
    "outputId": "50232b80-60ec-4e46-a1a0-d9a3e1c7780e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "def normalize(x):\n",
    "    \"\"\"\n",
    "    Normalize a list of sample image data in the range of 0 to 1\n",
    "    : x: List of image data.  The image shape is (32, 32, 3)\n",
    "    : return: Numpy array of normalize data\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    return x / 255.0\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_normalize(normalize)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "uQIqUDj2yGUU"
   },
   "source": [
    "### One-hot 编码\n",
    "\n",
    "和之前的代码单元一样，你将为预处理实现一个函数。这次，你将实现 `one_hot_encode` 函数。输入，也就是 `x`，是一个标签列表。实现该函数，以返回为 one_hot 编码的 Numpy 数组的标签列表。标签的可能值为 0 到 9。每次调用 `one_hot_encode` 时，对于每个值，one_hot 编码函数应该返回相同的编码。确保将编码映射保存到该函数外面。\n",
    "\n",
    "提示：不要重复发明轮子。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 35,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 641,
     "status": "ok",
     "timestamp": 1518436308981,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "X1TDIeffyGUU",
    "outputId": "40e5391e-40ba-4bcf-c582-297ba54a02f1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import label_binarize\n",
    "# http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.label_binarize.html#sklearn.preprocessing.label_binarize\n",
    "label_one_hot_classes = range(10)\n",
    "def one_hot_encode(x):\n",
    "    \"\"\"\n",
    "    One hot encode a list of sample labels. Return a one-hot encoded vector for each label.\n",
    "    : x: List of sample Labels\n",
    "    : return: Numpy array of one-hot encoded labels\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    # print(label_binarize(x, classes = label_one_hot_classes))\n",
    "    return label_binarize(x, classes = label_one_hot_classes)\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_one_hot_encode(one_hot_encode)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "KYzxIP8VyGUY"
   },
   "source": [
    "### 随机化数据\n",
    "\n",
    "之前探索数据时，你已经了解到，样本的顺序是随机的。再随机化一次也不会有什么关系，但是对于这个数据集没有必要。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "o51rb2PKyGUZ"
   },
   "source": [
    "## 预处理所有数据并保存\n",
    "\n",
    "运行下方的代码单元，将预处理所有 CIFAR-10 数据，并保存到文件中。下面的代码还使用了 10% 的训练数据，用来验证。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "CyQTIXCLyGUa"
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "# Preprocess Training, Validation, and Testing Data\n",
    "helper.preprocess_and_save_data(cifar10_dataset_folder_path, normalize, one_hot_encode)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "shtEdkfxyGUc"
   },
   "source": [
    "# 检查点\n",
    "\n",
    "这是你的第一个检查点。如果你什么时候决定再回到该记事本，或需要重新启动该记事本，你可以从这里开始。预处理的数据已保存到本地。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "9rouMLeqyGUd"
   },
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "import pickle\n",
    "import problem_unittests as tests\n",
    "import helper\n",
    "\n",
    "# Load the Preprocessed Validation data\n",
    "valid_features, valid_labels = pickle.load(open('preprocess_validation.p', mode='rb'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "T_DzPEmsyGUf"
   },
   "source": [
    "## 构建网络\n",
    "\n",
    "对于该神经网络，你需要将每层都构建为一个函数。你看到的大部分代码都位于函数外面。要更全面地测试你的代码，我们需要你将每层放入一个函数中。这样使我们能够提供更好的反馈，并使用我们的统一测试检测简单的错误，然后再提交项目。\n",
    "\n",
    ">**注意**：如果你觉得每周很难抽出足够的时间学习这门课程，我们为此项目提供了一个小捷径。对于接下来的几个问题，你可以使用 [TensorFlow Layers](https://www.tensorflow.org/api_docs/python/tf/layers) 或 [TensorFlow Layers (contrib)](https://www.tensorflow.org/api_guides/python/contrib.layers) 程序包中的类来构建每个层级，但是“卷积和最大池化层级”部分的层级除外。TF Layers 和 Keras 及 TFLearn 层级类似，因此很容易学会。\n",
    "\n",
    ">但是，如果你想充分利用这门课程，请尝试自己解决所有问题，不使用 TF Layers 程序包中的任何类。你依然可以使用其他程序包中的类，这些类和你在 TF Layers 中的类名称是一样的！例如，你可以使用 TF Neural Network 版本的 `conv2d` 类 [tf.nn.conv2d](https://www.tensorflow.org/api_docs/python/tf/nn/conv2d)，而不是 TF Layers 版本的 `conv2d` 类 [tf.layers.conv2d](https://www.tensorflow.org/api_docs/python/tf/layers/conv2d)。\n",
    "\n",
    "我们开始吧！\n",
    "\n",
    "\n",
    "### 输入\n",
    "\n",
    "神经网络需要读取图片数据、one-hot 编码标签和丢弃保留概率（dropout keep probability）。请实现以下函数：\n",
    "\n",
    "* 实现 `neural_net_image_input`\n",
    " * 返回 [TF Placeholder](https://www.tensorflow.org/api_docs/python/tf/placeholder)\n",
    " * 使用 `image_shape` 设置形状，部分大小设为 `None`\n",
    " * 使用 [TF Placeholder](https://www.tensorflow.org/api_docs/python/tf/placeholder) 中的 TensorFlow `name` 参数对 TensorFlow 占位符 \"x\" 命名\n",
    "* 实现 `neural_net_label_input`\n",
    " * 返回 [TF Placeholder](https://www.tensorflow.org/api_docs/python/tf/placeholder)\n",
    " * 使用 `n_classes` 设置形状，部分大小设为 `None`\n",
    " * 使用 [TF Placeholder](https://www.tensorflow.org/api_docs/python/tf/placeholder) 中的 TensorFlow `name` 参数对 TensorFlow 占位符 \"y\" 命名\n",
    "* 实现 `neural_net_keep_prob_input`\n",
    " * 返回 [TF Placeholder](https://www.tensorflow.org/api_docs/python/tf/placeholder)，用于丢弃保留概率\n",
    " * 使用 [TF Placeholder](https://www.tensorflow.org/api_docs/python/tf/placeholder) 中的 TensorFlow `name` 参数对 TensorFlow 占位符 \"keep_prob\" 命名\n",
    "\n",
    "这些名称将在项目结束时，用于加载保存的模型。\n",
    "\n",
    "注意：TensorFlow 中的 `None` 表示形状可以是动态大小。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 72,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 646,
     "status": "ok",
     "timestamp": 1518436326989,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "Iz6FEmwvyGUg",
    "outputId": "80f0ae1e-e486-4bb1-9178-49c95f268b39"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Image Input Tests Passed.\n",
      "Label Input Tests Passed.\n",
      "Keep Prob Tests Passed.\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "def neural_net_image_input(image_shape):\n",
    "    \"\"\"\n",
    "    Return a Tensor for a batch of image input\n",
    "    : image_shape: Shape of the images\n",
    "    : return: Tensor for image input.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "\n",
    "    return tf.placeholder(tf.float32, shape = [None, *image_shape], name = \"x\")\n",
    "\n",
    "\n",
    "def neural_net_label_input(n_classes):\n",
    "    \"\"\"\n",
    "    Return a Tensor for a batch of label input\n",
    "    : n_classes: Number of classes\n",
    "    : return: Tensor for label input.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    return tf.placeholder(tf.float32, shape = [None, n_classes], name = \"y\")\n",
    "\n",
    "\n",
    "def neural_net_keep_prob_input():\n",
    "    \"\"\"\n",
    "    Return a Tensor for keep probability\n",
    "    : return: Tensor for keep probability.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    return tf.placeholder(tf.float32, shape = None, name = \"keep_prob\")\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tf.reset_default_graph()\n",
    "tests.test_nn_image_inputs(neural_net_image_input)\n",
    "tests.test_nn_label_inputs(neural_net_label_input)\n",
    "tests.test_nn_keep_prob_inputs(neural_net_keep_prob_input)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "wpi6NLnCyGUj"
   },
   "source": [
    "### 卷积和最大池化层\n",
    "\n",
    "卷积层级适合处理图片。对于此代码单元，你应该实现函数 `conv2d_maxpool` 以便应用卷积然后进行最大池化：\n",
    "\n",
    "* 使用 `conv_ksize`、`conv_num_outputs` 和 `x_tensor` 的形状创建权重（weight）和偏置（bias）。\n",
    "* 使用权重和 `conv_strides` 对 `x_tensor` 应用卷积。\n",
    " * 建议使用我们建议的间距（padding），当然也可以使用任何其他间距。\n",
    "* 添加偏置\n",
    "* 向卷积中添加非线性激活（nonlinear activation）\n",
    "* 使用 `pool_ksize` 和 `pool_strides` 应用最大池化\n",
    " * 建议使用我们建议的间距（padding），当然也可以使用任何其他间距。\n",
    "\n",
    "**注意**：对于**此层**，**请勿使用** [TensorFlow Layers](https://www.tensorflow.org/api_docs/python/tf/layers) 或 [TensorFlow Layers (contrib)](https://www.tensorflow.org/api_guides/python/contrib.layers)，但是仍然可以使用 TensorFlow 的 [Neural Network](https://www.tensorflow.org/api_docs/python/tf/nn) 包。对于所有**其他层**，你依然可以使用快捷方法。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 35,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 623,
     "status": "ok",
     "timestamp": 1518436328169,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "51zNQAymyGUl",
    "outputId": "56ae3af1-32e4-48e7-c143-1522b01006f1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides):\n",
    "    \"\"\"\n",
    "    Apply convolution then max pooling to x_tensor\n",
    "    :param x_tensor: TensorFlow Tensor\n",
    "    :param conv_num_outputs: Number of outputs for the convolutional layer\n",
    "    :param conv_ksize: kernal size 2-D Tuple for the convolutional layer\n",
    "    :param conv_strides: Stride 2-D Tuple for convolution\n",
    "    :param pool_ksize: kernal size 2-D Tuple for pool\n",
    "    :param pool_strides: Stride 2-D Tuple for pool\n",
    "    : return: A tensor that represents convolution and max pooling of x_tensor\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    W_conv = tf.Variable(\n",
    "        tf.truncated_normal(\n",
    "            shape = [*conv_ksize,int(x_tensor.shape[3]), conv_num_outputs],\n",
    "            stddev=0.1))\n",
    "    b_conv = tf.Variable(tf.constant(0.1, shape = [conv_num_outputs]))\n",
    "    h_conv = tf.nn.relu(\n",
    "        tf.nn.conv2d(x_tensor, W_conv, [1, *conv_strides, 1], padding = \"SAME\")\n",
    "        + b_conv)\n",
    "    h_pool = tf.nn.max_pool(h_conv, [1, *pool_ksize, 1], [1, *pool_strides, 1], padding = \"SAME\")\n",
    "    return h_pool\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_con_pool(conv2d_maxpool)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4Cx6_6sgyGUo"
   },
   "source": [
    "### 扁平化层\n",
    "\n",
    "实现 `flatten` 函数，将 `x_tensor` 的维度从四维张量（4-D tensor）变成二维张量。输出应该是形状（*部分大小（Batch Size）*，*扁平化图片大小（Flattened Image Size）*）。快捷方法：对于此层，你可以使用 [TensorFlow Layers](https://www.tensorflow.org/api_docs/python/tf/layers) 或 [TensorFlow Layers (contrib)](https://www.tensorflow.org/api_guides/python/contrib.layers) 包中的类。如果你想要更大挑战，可以仅使用其他 TensorFlow 程序包。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 35,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 666,
     "status": "ok",
     "timestamp": 1518436329193,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "e43rY4hByGUp",
    "outputId": "2ecf0844-5f52-4de9-d191-654c5b8e8ed5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def flatten(x_tensor):\n",
    "    \"\"\"\n",
    "    Flatten x_tensor to (Batch Size, Flattened Image Size)\n",
    "    : x_tensor: A tensor of size (Batch Size, ...), where ... are the image dimensions.\n",
    "    : return: A tensor of size (Batch Size, Flattened Image Size).\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    flattened_image_size = int(x_tensor.shape[1] \n",
    "                               * x_tensor.shape[2]\n",
    "                               * x_tensor.shape[3])\n",
    "    return tf.reshape(x_tensor, shape = [-1, flattened_image_size])\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_flatten(flatten)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "I46_A-WMyGUu"
   },
   "source": [
    "### 全连接层\n",
    "\n",
    "实现 `fully_conn` 函数，以向 `x_tensor` 应用完全连接的层级，形状为（*部分大小（Batch Size）*，*num_outputs*）。快捷方法：对于此层，你可以使用 [TensorFlow Layers](https://www.tensorflow.org/api_docs/python/tf/layers) 或 [TensorFlow Layers (contrib)](https://www.tensorflow.org/api_guides/python/contrib.layers) 包中的类。如果你想要更大挑战，可以仅使用其他 TensorFlow 程序包。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 35,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 599,
     "status": "ok",
     "timestamp": 1518436330029,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "H1plqCRZyGUw",
    "outputId": "b17ce661-a2eb-427d-9bcc-2db7b01be876"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def fully_conn(x_tensor, num_outputs):\n",
    "    \"\"\"\n",
    "    Apply a fully connected layer to x_tensor using weight and bias\n",
    "    : x_tensor: A 2-D tensor where the first dimension is batch size.\n",
    "    : num_outputs: The number of output that the new tensor should be.\n",
    "    : return: A 2-D tensor where the second dimension is num_outputs.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    W_fc = tf.Variable(\n",
    "        tf.truncated_normal(\n",
    "            shape = [int(x_tensor.shape[1]), num_outputs],\n",
    "            stddev = 0.1))\n",
    "    b_fc = tf.Variable(tf.constant(0.1, shape = [num_outputs]))\n",
    "    h_fc = tf.nn.relu(tf.matmul(x_tensor, W_fc) + b_fc)\n",
    "    return h_fc\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_fully_conn(fully_conn)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gzdpCW-EyGU0"
   },
   "source": [
    "### 输出层\n",
    "\n",
    "实现 `output` 函数，向 x_tensor 应用完全连接的层级，形状为（*部分大小（Batch Size）*，*num_outputs*）。快捷方法：对于此层，你可以使用 [TensorFlow Layers](https://www.tensorflow.org/api_docs/python/tf/layers) 或 [TensorFlow Layers (contrib)](https://www.tensorflow.org/api_guides/python/contrib.layers) 包中的类。如果你想要更大挑战，可以仅使用其他 TensorFlow 程序包。\n",
    "\n",
    "**注意**：该层级不应应用 Activation、softmax 或交叉熵（cross entropy）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 35,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 694,
     "status": "ok",
     "timestamp": 1518436330840,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "WlONY3L8yGU1",
    "outputId": "c51b9398-2a1b-4a4f-e6bd-5ad73eb9ea9c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def output(x_tensor, num_outputs):\n",
    "    \"\"\"\n",
    "    Apply a output layer to x_tensor using weight and bias\n",
    "    : x_tensor: A 2-D tensor where the first dimension is batch size.\n",
    "    : num_outputs: The number of output that the new tensor should be.\n",
    "    : return: A 2-D tensor where the second dimension is num_outputs.\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    W_fc = tf.Variable(\n",
    "        tf.truncated_normal(\n",
    "            shape = [int(x_tensor.shape[1]), num_outputs],\n",
    "            stddev = 0.1))\n",
    "    b_fc = tf.Variable(tf.constant(0.1, shape = [num_outputs]))\n",
    "    h_fc = tf.matmul(x_tensor, W_fc) + b_fc\n",
    "    return h_fc\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_output(output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "E8VpatBFyGU5"
   },
   "source": [
    "### 创建卷积模型\n",
    "\n",
    "实现函数 `conv_net`， 创建卷积神经网络模型。该函数传入一批图片 `x`，并输出对数（logits）。使用你在上方创建的层创建此模型：\n",
    "\n",
    "* 应用 1、2 或 3 个卷积和最大池化层（Convolution and Max Pool layers）\n",
    "* 应用一个扁平层（Flatten Layer）\n",
    "* 应用 1、2 或 3 个完全连接层（Fully Connected Layers）\n",
    "* 应用一个输出层（Output Layer）\n",
    "* 返回输出\n",
    "* 使用 `keep_prob` 向模型中的一个或多个层应用 [TensorFlow 的 Dropout](https://www.tensorflow.org/api_docs/python/tf/nn/dropout)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 35,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 946,
     "status": "ok",
     "timestamp": 1518436331951,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "b0uXvFObyGU5",
    "outputId": "350c3e10-e120-46df-c0a3-3765369c3022"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Neural Network Built!\n"
     ]
    }
   ],
   "source": [
    "def conv_net(x, keep_prob):\n",
    "    \"\"\"\n",
    "    Create a convolutional neural network model\n",
    "    : x: Placeholder tensor that holds image data.\n",
    "    : keep_prob: Placeholder tensor that hold dropout keep probability.\n",
    "    : return: Tensor that represents logits\n",
    "    \"\"\"\n",
    "    # TODO: Apply 1, 2, or 3 Convolution and Max Pool layers\n",
    "    #    Play around with different number of outputs, kernel size and stride\n",
    "    # Function Definition from Above:\n",
    "    #    conv2d_maxpool(x_tensor, conv_num_outputs, conv_ksize, conv_strides, pool_ksize, pool_strides)\n",
    "    conv1 = conv2d_maxpool(x, 32, [3, 3], [1, 1], [2, 2], [2, 2])\n",
    "    conv1_drop = tf.nn.dropout(conv1, keep_prob)\n",
    "    conv2 = conv2d_maxpool(conv1_drop, 64, [3, 3], [1, 1], [2, 2], [2, 2])\n",
    "    conv2_drop = tf.nn.dropout(conv2, keep_prob)\n",
    "    \n",
    "\n",
    "    # TODO: Apply a Flatten Layer\n",
    "    # Function Definition from Above:\n",
    "    #   flatten(x_tensor)\n",
    "    flat1 = flatten(conv2_drop)\n",
    "    \n",
    "    flat1_drop = tf.nn.dropout(flat1, keep_prob)\n",
    "    \n",
    "\n",
    "    # TODO: Apply 1, 2, or 3 Fully Connected Layers\n",
    "    #    Play around with different number of outputs\n",
    "    # Function Definition from Above:\n",
    "    #   fully_conn(x_tensor, num_outputs)\n",
    "    fc1 = fully_conn(flat1_drop, 512)\n",
    "    fc1_drop = tf.nn.dropout(fc1, keep_prob)\n",
    "    \n",
    "    # TODO: Apply an Output Layer\n",
    "    #    Set this to the number of classes\n",
    "    # Function Definition from Above:\n",
    "    #   output(x_tensor, num_outputs)\n",
    "    y_out = output(fc1_drop, 10)\n",
    "    \n",
    "    \n",
    "    # TODO: return output\n",
    "    return y_out\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "\n",
    "##############################\n",
    "## Build the Neural Network ##\n",
    "##############################\n",
    "\n",
    "# Remove previous weights, bias, inputs, etc..\n",
    "tf.reset_default_graph()\n",
    "\n",
    "# Inputs\n",
    "x = neural_net_image_input((32, 32, 3))\n",
    "y = neural_net_label_input(10)\n",
    "keep_prob = neural_net_keep_prob_input()\n",
    "\n",
    "# Model\n",
    "logits = conv_net(x, keep_prob)\n",
    "\n",
    "# Name logits Tensor, so that is can be loaded from disk after training\n",
    "logits = tf.identity(logits, name='logits')\n",
    "\n",
    "# Loss and Optimizer\n",
    "cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))\n",
    "optimizer = tf.train.AdamOptimizer().minimize(cost)\n",
    "\n",
    "# Accuracy\n",
    "correct_pred = tf.equal(tf.argmax(logits, 1), tf.argmax(y, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32), name='accuracy')\n",
    "\n",
    "tests.test_conv_net(conv_net)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XGJUPpPkyGU9"
   },
   "source": [
    "## 训练神经网络\n",
    "\n",
    "### 单次优化\n",
    "\n",
    "实现函数 `train_neural_network` 以进行单次优化（single optimization）。该优化应该使用 `optimizer` 优化 `session`，其中 `feed_dict` 具有以下参数：\n",
    "\n",
    "* `x` 表示图片输入\n",
    "* `y` 表示标签\n",
    "* `keep_prob` 表示丢弃的保留率\n",
    "\n",
    "每个部分都会调用该函数，所以 `tf.global_variables_initializer()` 已经被调用。\n",
    "\n",
    "注意：不需要返回任何内容。该函数只是用来优化神经网络。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 35,
     "output_extras": [
      {
       "item_id": 1
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 617,
     "status": "ok",
     "timestamp": 1518436332774,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "feVJ5DoayGU_",
    "outputId": "bd475178-0811-4d55-b877-ba7f05b2b4e2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tests Passed\n"
     ]
    }
   ],
   "source": [
    "def train_neural_network(session, optimizer, keep_probability, feature_batch, label_batch):\n",
    "    \"\"\"\n",
    "    Optimize the session on a batch of images and labels\n",
    "    : session: Current TensorFlow session\n",
    "    : optimizer: TensorFlow optimizer function\n",
    "    : keep_probability: keep probability\n",
    "    : feature_batch: Batch of Numpy image data\n",
    "    : label_batch: Batch of Numpy label data\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    session.run(optimizer, feed_dict = {x: feature_batch, y: label_batch, keep_prob: keep_probability})\n",
    "    pass\n",
    "\n",
    "\n",
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL THAT IS BELOW THIS LINE\n",
    "\"\"\"\n",
    "tests.test_train_nn(train_neural_network)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dll3HzOeyGVD"
   },
   "source": [
    "### 显示数据\n",
    "\n",
    "实现函数 `print_stats` 以输出损失和验证准确率。使用全局变量 `valid_features` 和 `valid_labels` 计算验证准确率。使用保留率 `1.0` 计算损失和验证准确率（loss and validation accuracy）。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "Wnt0ULMzyGVD"
   },
   "outputs": [],
   "source": [
    "def print_stats(session, feature_batch, label_batch, cost, accuracy):\n",
    "    \"\"\"\n",
    "    Print information about loss and validation accuracy\n",
    "    : session: Current TensorFlow session\n",
    "    : feature_batch: Batch of Numpy image data\n",
    "    : label_batch: Batch of Numpy label data\n",
    "    : cost: TensorFlow cost function\n",
    "    : accuracy: TensorFlow accuracy function\n",
    "    \"\"\"\n",
    "    # TODO: Implement Function\n",
    "    print(\" loss: %g\"%session.run(cost, feed_dict = {x: feature_batch, y: label_batch, keep_prob:1.0}), end='')\n",
    "    print(\" accuracy: %g\"%session.run(accuracy, feed_dict = {x: valid_features, y: valid_labels, keep_prob: 1.0}))\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "icCHZYDcyGVF"
   },
   "source": [
    "### 超参数\n",
    "\n",
    "调试以下超参数：\n",
    "* 设置 `epochs` 表示神经网络停止学习或开始过拟合的迭代次数\n",
    "* 设置 `batch_size`，表示机器内存允许的部分最大体积。大部分人设为以下常见内存大小：\n",
    "\n",
    " * 64\n",
    " * 128\n",
    " * 256\n",
    " * ...\n",
    "* 设置 `keep_probability` 表示使用丢弃时保留节点的概率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     }
    },
    "colab_type": "code",
    "id": "vBfW40k3yGVH"
   },
   "outputs": [],
   "source": [
    "# TODO: Tune Parameters\n",
    "epochs = 200\n",
    "batch_size = 512\n",
    "keep_probability = 0.4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "c4beir69yGVK"
   },
   "source": [
    "### 在单个 CIFAR-10 部分上训练\n",
    "\n",
    "我们先用单个部分，而不是用所有的 CIFAR-10 批次训练神经网络。这样可以节省时间，并对模型进行迭代，以提高准确率。最终验证准确率达到 50% 或以上之后，在下一部分对所有数据运行模型。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 3728,
     "output_extras": [
      {
       "item_id": 58
      },
      {
       "item_id": 114
      },
      {
       "item_id": 170
      },
      {
       "item_id": 202
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 238016,
     "status": "ok",
     "timestamp": 1518436573025,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "7u6K4VVjyGVL",
    "outputId": "dcbac6e3-5345-47e5-d137-947136d35681"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checking the Training on a Single Batch...\n",
      "Epoch  1, CIFAR-10 Batch 1:   loss: 2.30167 accuracy: 0.1004\n",
      "Epoch  2, CIFAR-10 Batch 1:   loss: 2.28667 accuracy: 0.131\n",
      "Epoch  3, CIFAR-10 Batch 1:   loss: 2.28462 accuracy: 0.1684\n",
      "Epoch  4, CIFAR-10 Batch 1:   loss: 2.29135 accuracy: 0.1712\n",
      "Epoch  5, CIFAR-10 Batch 1:   loss: 2.29732 accuracy: 0.1472\n",
      "Epoch  6, CIFAR-10 Batch 1:   loss: 2.29903 accuracy: 0.129\n",
      "Epoch  7, CIFAR-10 Batch 1:   loss: 2.29951 accuracy: 0.1176\n",
      "Epoch  8, CIFAR-10 Batch 1:   loss: 2.29948 accuracy: 0.1288\n",
      "Epoch  9, CIFAR-10 Batch 1:   loss: 2.29919 accuracy: 0.138\n",
      "Epoch 10, CIFAR-10 Batch 1:   loss: 2.29733 accuracy: 0.1372\n",
      "Epoch 11, CIFAR-10 Batch 1:   loss: 2.29138 accuracy: 0.1538\n",
      "Epoch 12, CIFAR-10 Batch 1:   loss: 2.2768 accuracy: 0.1828\n",
      "Epoch 13, CIFAR-10 Batch 1:   loss: 2.25733 accuracy: 0.1844\n",
      "Epoch 14, CIFAR-10 Batch 1:   loss: 2.2289 accuracy: 0.2082\n",
      "Epoch 15, CIFAR-10 Batch 1:   loss: 2.19219 accuracy: 0.2134\n",
      "Epoch 16, CIFAR-10 Batch 1:   loss: 2.15017 accuracy: 0.2196\n",
      "Epoch 17, CIFAR-10 Batch 1:   loss: 2.11773 accuracy: 0.2254\n",
      "Epoch 18, CIFAR-10 Batch 1:   loss: 2.11732 accuracy: 0.2394\n",
      "Epoch 19, CIFAR-10 Batch 1:   loss: 2.08702 accuracy: 0.2502\n",
      "Epoch 20, CIFAR-10 Batch 1:   loss: 2.08225 accuracy: 0.2584\n",
      "Epoch 21, CIFAR-10 Batch 1:   loss: 2.06308 accuracy: 0.2738\n",
      "Epoch 22, CIFAR-10 Batch 1:   loss: 2.07221 accuracy: 0.2882\n",
      "Epoch 23, CIFAR-10 Batch 1:   loss: 2.04206 accuracy: 0.2826\n",
      "Epoch 24, CIFAR-10 Batch 1:   loss: 2.03261 accuracy: 0.2954\n",
      "Epoch 25, CIFAR-10 Batch 1:   loss: 2.02823 accuracy: 0.2922\n",
      "Epoch 26, CIFAR-10 Batch 1:   loss: 2.00995 accuracy: 0.3002\n",
      "Epoch 27, CIFAR-10 Batch 1:   loss: 2.01607 accuracy: 0.3136\n",
      "Epoch 28, CIFAR-10 Batch 1:   loss: 1.99717 accuracy: 0.3142\n",
      "Epoch 29, CIFAR-10 Batch 1:   loss: 1.9768 accuracy: 0.3212\n",
      "Epoch 30, CIFAR-10 Batch 1:   loss: 1.96959 accuracy: 0.319\n",
      "Epoch 31, CIFAR-10 Batch 1:   loss: 1.98145 accuracy: 0.3334\n",
      "Epoch 32, CIFAR-10 Batch 1:   loss: 1.95909 accuracy: 0.3294\n",
      "Epoch 33, CIFAR-10 Batch 1:   loss: 1.94869 accuracy: 0.3316\n",
      "Epoch 34, CIFAR-10 Batch 1:   loss: 1.93054 accuracy: 0.339\n",
      "Epoch 35, CIFAR-10 Batch 1:   loss: 1.93747 accuracy: 0.3402\n",
      "Epoch 36, CIFAR-10 Batch 1:   loss: 1.91639 accuracy: 0.3396\n",
      "Epoch 37, CIFAR-10 Batch 1:   loss: 1.92226 accuracy: 0.3468\n",
      "Epoch 38, CIFAR-10 Batch 1:   loss: 1.9109 accuracy: 0.3448\n",
      "Epoch 39, CIFAR-10 Batch 1:   loss: 1.89784 accuracy: 0.354\n",
      "Epoch 40, CIFAR-10 Batch 1:   loss: 1.89242 accuracy: 0.3578\n",
      "Epoch 41, CIFAR-10 Batch 1:   loss: 1.902 accuracy: 0.3608\n",
      "Epoch 42, CIFAR-10 Batch 1:   loss: 1.88007 accuracy: 0.3576\n",
      "Epoch 43, CIFAR-10 Batch 1:   loss: 1.86861 accuracy: 0.3626\n",
      "Epoch 44, CIFAR-10 Batch 1:   loss: 1.85188 accuracy: 0.3682\n",
      "Epoch 45, CIFAR-10 Batch 1:   loss: 1.85128 accuracy: 0.3706\n",
      "Epoch 46, CIFAR-10 Batch 1:   loss: 1.83883 accuracy: 0.3736\n",
      "Epoch 47, CIFAR-10 Batch 1:   loss: 1.83793 accuracy: 0.3744\n",
      "Epoch 48, CIFAR-10 Batch 1:   loss: 1.85001 accuracy: 0.3744\n",
      "Epoch 49, CIFAR-10 Batch 1:   loss: 1.81431 accuracy: 0.3776\n",
      "Epoch 50, CIFAR-10 Batch 1:   loss: 1.81803 accuracy: 0.3852\n",
      "Epoch 51, CIFAR-10 Batch 1:   loss: 1.80127 accuracy: 0.387\n",
      "Epoch 52, CIFAR-10 Batch 1:   loss: 1.81871 accuracy: 0.3912\n",
      "Epoch 53, CIFAR-10 Batch 1:   loss: 1.79564 accuracy: 0.3874\n",
      "Epoch 54, CIFAR-10 Batch 1:   loss: 1.8025 accuracy: 0.3918\n",
      "Epoch 55, CIFAR-10 Batch 1:   loss: 1.78204 accuracy: 0.395\n",
      "Epoch 56, CIFAR-10 Batch 1:   loss: 1.77635 accuracy: 0.3948\n",
      "Epoch 57, CIFAR-10 Batch 1:   loss: 1.78709 accuracy: 0.3942\n",
      "Epoch 58, CIFAR-10 Batch 1:   loss: 1.76363 accuracy: 0.388\n",
      "Epoch 59, CIFAR-10 Batch 1:   loss: 1.76854 accuracy: 0.3916\n",
      "Epoch 60, CIFAR-10 Batch 1:   loss: 1.75455 accuracy: 0.3916\n",
      "Epoch 61, CIFAR-10 Batch 1:   loss: 1.73582 accuracy: 0.398\n",
      "Epoch 62, CIFAR-10 Batch 1:   loss: 1.73396 accuracy: 0.4008\n",
      "Epoch 63, CIFAR-10 Batch 1:   loss: 1.75261 accuracy: 0.3988\n",
      "Epoch 64, CIFAR-10 Batch 1:   loss: 1.7373 accuracy: 0.4012\n",
      "Epoch 65, CIFAR-10 Batch 1:   loss: 1.71939 accuracy: 0.4026\n",
      "Epoch 66, CIFAR-10 Batch 1:   loss: 1.72314 accuracy: 0.4112\n",
      "Epoch 67, CIFAR-10 Batch 1:   loss: 1.72613 accuracy: 0.4054\n",
      "Epoch 68, CIFAR-10 Batch 1:   loss: 1.72099 accuracy: 0.4014\n",
      "Epoch 69, CIFAR-10 Batch 1:   loss: 1.70187 accuracy: 0.4138\n",
      "Epoch 70, CIFAR-10 Batch 1:   loss: 1.71286 accuracy: 0.408\n",
      "Epoch 71, CIFAR-10 Batch 1:   loss: 1.66873 accuracy: 0.4206\n",
      "Epoch 72, CIFAR-10 Batch 1:   loss: 1.67089 accuracy: 0.4178\n",
      "Epoch 73, CIFAR-10 Batch 1:   loss: 1.65431 accuracy: 0.4216\n",
      "Epoch 74, CIFAR-10 Batch 1:   loss: 1.6497 accuracy: 0.4292\n",
      "Epoch 75, CIFAR-10 Batch 1:   loss: 1.6425 accuracy: 0.425\n",
      "Epoch 76, CIFAR-10 Batch 1:   loss: 1.64265 accuracy: 0.4256\n",
      "Epoch 77, CIFAR-10 Batch 1:   loss: 1.6408 accuracy: 0.432\n",
      "Epoch 78, CIFAR-10 Batch 1:   loss: 1.64842 accuracy: 0.432\n",
      "Epoch 79, CIFAR-10 Batch 1:   loss: 1.63273 accuracy: 0.442\n",
      "Epoch 80, CIFAR-10 Batch 1:   loss: 1.63687 accuracy: 0.4314\n",
      "Epoch 81, CIFAR-10 Batch 1:   loss: 1.61716 accuracy: 0.437\n",
      "Epoch 82, CIFAR-10 Batch 1:   loss: 1.60133 accuracy: 0.4368\n",
      "Epoch 83, CIFAR-10 Batch 1:   loss: 1.62857 accuracy: 0.437\n",
      "Epoch 84, CIFAR-10 Batch 1:   loss: 1.5995 accuracy: 0.4406\n",
      "Epoch 85, CIFAR-10 Batch 1:   loss: 1.59755 accuracy: 0.4444\n",
      "Epoch 86, CIFAR-10 Batch 1:   loss: 1.59774 accuracy: 0.4456\n",
      "Epoch 87, CIFAR-10 Batch 1:   loss: 1.58902 accuracy: 0.4444\n",
      "Epoch 88, CIFAR-10 Batch 1:   loss: 1.56697 accuracy: 0.45\n",
      "Epoch 89, CIFAR-10 Batch 1:   loss: 1.57257 accuracy: 0.4558\n",
      "Epoch 90, CIFAR-10 Batch 1:   loss: 1.5645 accuracy: 0.4488\n",
      "Epoch 91, CIFAR-10 Batch 1:   loss: 1.57058 accuracy: 0.4534\n",
      "Epoch 92, CIFAR-10 Batch 1:   loss: 1.55101 accuracy: 0.456\n",
      "Epoch 93, CIFAR-10 Batch 1:   loss: 1.55712 accuracy: 0.4552\n",
      "Epoch 94, CIFAR-10 Batch 1:   loss: 1.53586 accuracy: 0.46\n",
      "Epoch 95, CIFAR-10 Batch 1:   loss: 1.53425 accuracy: 0.456\n",
      "Epoch 96, CIFAR-10 Batch 1:   loss: 1.52922 accuracy: 0.4642\n",
      "Epoch 97, CIFAR-10 Batch 1:   loss: 1.52052 accuracy: 0.4628\n",
      "Epoch 98, CIFAR-10 Batch 1:   loss: 1.51902 accuracy: 0.4672\n",
      "Epoch 99, CIFAR-10 Batch 1:   loss: 1.51393 accuracy: 0.4526\n",
      "Epoch 100, CIFAR-10 Batch 1:   loss: 1.52602 accuracy: 0.4666\n",
      "Epoch 101, CIFAR-10 Batch 1:   loss: 1.51782 accuracy: 0.4654\n",
      "Epoch 102, CIFAR-10 Batch 1:   loss: 1.51258 accuracy: 0.4682\n",
      "Epoch 103, CIFAR-10 Batch 1:   loss: 1.49892 accuracy: 0.465\n",
      "Epoch 104, CIFAR-10 Batch 1:   loss: 1.48452 accuracy: 0.4682\n",
      "Epoch 105, CIFAR-10 Batch 1:   loss: 1.47995 accuracy: 0.4744\n",
      "Epoch 106, CIFAR-10 Batch 1:   loss: 1.46738 accuracy: 0.4688\n",
      "Epoch 107, CIFAR-10 Batch 1:   loss: 1.46025 accuracy: 0.482\n",
      "Epoch 108, CIFAR-10 Batch 1:   loss: 1.45367 accuracy: 0.4844\n",
      "Epoch 109, CIFAR-10 Batch 1:   loss: 1.45854 accuracy: 0.473\n",
      "Epoch 110, CIFAR-10 Batch 1:   loss: 1.45086 accuracy: 0.48\n",
      "Epoch 111, CIFAR-10 Batch 1:   loss: 1.45694 accuracy: 0.4868\n",
      "Epoch 112, CIFAR-10 Batch 1:   loss: 1.44371 accuracy: 0.4834\n",
      "Epoch 113, CIFAR-10 Batch 1:   loss: 1.47109 accuracy: 0.4826\n",
      "Epoch 114, CIFAR-10 Batch 1:   loss: 1.45264 accuracy: 0.4834\n",
      "Epoch 115, CIFAR-10 Batch 1:   loss: 1.43509 accuracy: 0.4888\n",
      "Epoch 116, CIFAR-10 Batch 1:   loss: 1.42119 accuracy: 0.4858\n",
      "Epoch 117, CIFAR-10 Batch 1:   loss: 1.4372 accuracy: 0.482\n",
      "Epoch 118, CIFAR-10 Batch 1:   loss: 1.40275 accuracy: 0.4894\n",
      "Epoch 119, CIFAR-10 Batch 1:   loss: 1.41139 accuracy: 0.4954\n",
      "Epoch 120, CIFAR-10 Batch 1:   loss: 1.4074 accuracy: 0.4908\n",
      "Epoch 121, CIFAR-10 Batch 1:   loss: 1.38674 accuracy: 0.4968\n",
      "Epoch 122, CIFAR-10 Batch 1:   loss: 1.39588 accuracy: 0.4928\n",
      "Epoch 123, CIFAR-10 Batch 1:   loss: 1.37238 accuracy: 0.5002\n",
      "Epoch 124, CIFAR-10 Batch 1:   loss: 1.38693 accuracy: 0.5\n",
      "Epoch 125, CIFAR-10 Batch 1:   loss: 1.38219 accuracy: 0.4928\n",
      "Epoch 126, CIFAR-10 Batch 1:   loss: 1.37787 accuracy: 0.5034\n",
      "Epoch 127, CIFAR-10 Batch 1:   loss: 1.36303 accuracy: 0.5034\n",
      "Epoch 128, CIFAR-10 Batch 1:   loss: 1.34883 accuracy: 0.5034\n",
      "Epoch 129, CIFAR-10 Batch 1:   loss: 1.35064 accuracy: 0.5024\n",
      "Epoch 130, CIFAR-10 Batch 1:   loss: 1.36705 accuracy: 0.5008\n",
      "Epoch 131, CIFAR-10 Batch 1:   loss: 1.34115 accuracy: 0.5094\n",
      "Epoch 132, CIFAR-10 Batch 1:   loss: 1.3336 accuracy: 0.5114\n",
      "Epoch 133, CIFAR-10 Batch 1:   loss: 1.34922 accuracy: 0.5048\n",
      "Epoch 134, CIFAR-10 Batch 1:   loss: 1.33156 accuracy: 0.5078\n",
      "Epoch 135, CIFAR-10 Batch 1:   loss: 1.34128 accuracy: 0.515\n",
      "Epoch 136, CIFAR-10 Batch 1:   loss: 1.32944 accuracy: 0.507\n",
      "Epoch 137, CIFAR-10 Batch 1:   loss: 1.32791 accuracy: 0.5114\n",
      "Epoch 138, CIFAR-10 Batch 1:   loss: 1.31454 accuracy: 0.5144\n",
      "Epoch 139, CIFAR-10 Batch 1:   loss: 1.32675 accuracy: 0.5116\n",
      "Epoch 140, CIFAR-10 Batch 1:   loss: 1.29822 accuracy: 0.5154\n",
      "Epoch 141, CIFAR-10 Batch 1:   loss: 1.29707 accuracy: 0.5192\n",
      "Epoch 142, CIFAR-10 Batch 1:   loss: 1.29636 accuracy: 0.5184\n",
      "Epoch 143, CIFAR-10 Batch 1:   loss: 1.29593 accuracy: 0.521\n",
      "Epoch 144, CIFAR-10 Batch 1:   loss: 1.3055 accuracy: 0.5254\n",
      "Epoch 145, CIFAR-10 Batch 1:   loss: 1.25817 accuracy: 0.5212\n",
      "Epoch 146, CIFAR-10 Batch 1:   loss: 1.27269 accuracy: 0.5154\n",
      "Epoch 147, CIFAR-10 Batch 1:   loss: 1.2857 accuracy: 0.5176\n",
      "Epoch 148, CIFAR-10 Batch 1:   loss: 1.26334 accuracy: 0.5194\n",
      "Epoch 149, CIFAR-10 Batch 1:   loss: 1.27236 accuracy: 0.5198\n",
      "Epoch 150, CIFAR-10 Batch 1:   loss: 1.27791 accuracy: 0.519\n",
      "Epoch 151, CIFAR-10 Batch 1:   loss: 1.26065 accuracy: 0.5244\n",
      "Epoch 152, CIFAR-10 Batch 1:   loss: 1.27092 accuracy: 0.5288\n",
      "Epoch 153, CIFAR-10 Batch 1:   loss: 1.27574 accuracy: 0.5214\n",
      "Epoch 154, CIFAR-10 Batch 1:   loss: 1.27128 accuracy: 0.5238\n",
      "Epoch 155, CIFAR-10 Batch 1:   loss: 1.24617 accuracy: 0.5194\n",
      "Epoch 156, CIFAR-10 Batch 1:   loss: 1.23735 accuracy: 0.5286\n",
      "Epoch 157, CIFAR-10 Batch 1:   loss: 1.21785 accuracy: 0.5292\n",
      "Epoch 158, CIFAR-10 Batch 1:   loss: 1.2313 accuracy: 0.531\n",
      "Epoch 159, CIFAR-10 Batch 1:   loss: 1.23573 accuracy: 0.5304\n",
      "Epoch 160, CIFAR-10 Batch 1:   loss: 1.2124 accuracy: 0.5336\n",
      "Epoch 161, CIFAR-10 Batch 1:   loss: 1.24471 accuracy: 0.527\n",
      "Epoch 162, CIFAR-10 Batch 1:   loss: 1.21848 accuracy: 0.54\n",
      "Epoch 163, CIFAR-10 Batch 1:   loss: 1.20272 accuracy: 0.5372\n",
      "Epoch 164, CIFAR-10 Batch 1:   loss: 1.21499 accuracy: 0.5342\n",
      "Epoch 165, CIFAR-10 Batch 1:   loss: 1.20165 accuracy: 0.5416\n",
      "Epoch 166, CIFAR-10 Batch 1:   loss: 1.1877 accuracy: 0.542\n",
      "Epoch 167, CIFAR-10 Batch 1:   loss: 1.20692 accuracy: 0.5362\n",
      "Epoch 168, CIFAR-10 Batch 1:   loss: 1.18196 accuracy: 0.5434\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 169, CIFAR-10 Batch 1:   loss: 1.1878 accuracy: 0.54\n",
      "Epoch 170, CIFAR-10 Batch 1:   loss: 1.17134 accuracy: 0.5464\n",
      "Epoch 171, CIFAR-10 Batch 1:   loss: 1.16019 accuracy: 0.5408\n",
      "Epoch 172, CIFAR-10 Batch 1:   loss: 1.16734 accuracy: 0.5484\n",
      "Epoch 173, CIFAR-10 Batch 1:   loss: 1.15001 accuracy: 0.5462\n",
      "Epoch 174, CIFAR-10 Batch 1:   loss: 1.15259 accuracy: 0.5496\n",
      "Epoch 175, CIFAR-10 Batch 1:   loss: 1.14325 accuracy: 0.5426\n",
      "Epoch 176, CIFAR-10 Batch 1:   loss: 1.16196 accuracy: 0.5432\n",
      "Epoch 177, CIFAR-10 Batch 1:   loss: 1.17856 accuracy: 0.5454\n",
      "Epoch 178, CIFAR-10 Batch 1:   loss: 1.15124 accuracy: 0.5438\n",
      "Epoch 179, CIFAR-10 Batch 1:   loss: 1.12978 accuracy: 0.5502\n",
      "Epoch 180, CIFAR-10 Batch 1:   loss: 1.14894 accuracy: 0.5412\n",
      "Epoch 181, CIFAR-10 Batch 1:   loss: 1.14525 accuracy: 0.5456\n",
      "Epoch 182, CIFAR-10 Batch 1:   loss: 1.13662 accuracy: 0.546\n",
      "Epoch 183, CIFAR-10 Batch 1:   loss: 1.13336 accuracy: 0.5538\n",
      "Epoch 184, CIFAR-10 Batch 1:   loss: 1.10819 accuracy: 0.556\n",
      "Epoch 185, CIFAR-10 Batch 1:   loss: 1.11953 accuracy: 0.561\n",
      "Epoch 186, CIFAR-10 Batch 1:   loss: 1.10255 accuracy: 0.5584\n",
      "Epoch 187, CIFAR-10 Batch 1:   loss: 1.10686 accuracy: 0.56\n",
      "Epoch 188, CIFAR-10 Batch 1:   loss: 1.11666 accuracy: 0.5572\n",
      "Epoch 189, CIFAR-10 Batch 1:   loss: 1.10553 accuracy: 0.5604\n",
      "Epoch 190, CIFAR-10 Batch 1:   loss: 1.10682 accuracy: 0.5618\n",
      "Epoch 191, CIFAR-10 Batch 1:   loss: 1.09477 accuracy: 0.5614\n",
      "Epoch 192, CIFAR-10 Batch 1:   loss: 1.0992 accuracy: 0.554\n",
      "Epoch 193, CIFAR-10 Batch 1:   loss: 1.08005 accuracy: 0.563\n",
      "Epoch 194, CIFAR-10 Batch 1:   loss: 1.09472 accuracy: 0.5604\n",
      "Epoch 195, CIFAR-10 Batch 1:   loss: 1.10951 accuracy: 0.5576\n",
      "Epoch 196, CIFAR-10 Batch 1:   loss: 1.10372 accuracy: 0.5624\n",
      "Epoch 197, CIFAR-10 Batch 1:   loss: 1.06376 accuracy: 0.5632\n",
      "Epoch 198, CIFAR-10 Batch 1:   loss: 1.07653 accuracy: 0.561\n",
      "Epoch 199, CIFAR-10 Batch 1:   loss: 1.07778 accuracy: 0.5672\n",
      "Epoch 200, CIFAR-10 Batch 1:   loss: 1.08624 accuracy: 0.5622\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "print('Checking the Training on a Single Batch...')\n",
    "with tf.Session() as sess:\n",
    "    # Initializing the variables\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    \n",
    "    # Training cycle\n",
    "    for epoch in range(epochs):\n",
    "        batch_i = 1\n",
    "        for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):\n",
    "            train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)\n",
    "        print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')\n",
    "        print_stats(sess, batch_features, batch_labels, cost, accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EZBPuDxgyGVQ"
   },
   "source": [
    "### 完全训练模型\n",
    "\n",
    "现在，单个 CIFAR-10 部分的准确率已经不错了，试试所有五个部分吧。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 18497,
     "output_extras": [
      {
       "item_id": 58
      },
      {
       "item_id": 114
      },
      {
       "item_id": 170
      },
      {
       "item_id": 226
      },
      {
       "item_id": 282
      },
      {
       "item_id": 338
      },
      {
       "item_id": 394
      },
      {
       "item_id": 450
      },
      {
       "item_id": 506
      },
      {
       "item_id": 561
      },
      {
       "item_id": 616
      },
      {
       "item_id": 671
      },
      {
       "item_id": 726
      },
      {
       "item_id": 781
      },
      {
       "item_id": 836
      },
      {
       "item_id": 891
      },
      {
       "item_id": 946
      },
      {
       "item_id": 1002
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 1191236,
     "status": "ok",
     "timestamp": 1518437764292,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "Wpez8I5fyGVQ",
    "outputId": "c6545410-15a1-430a-cf4b-6ad515331411"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training...\n",
      "Epoch  1, CIFAR-10 Batch 1:   loss: 2.30912 accuracy: 0.1108\n",
      "Epoch  1, CIFAR-10 Batch 2:   loss: 2.28775 accuracy: 0.1304\n",
      "Epoch  1, CIFAR-10 Batch 3:   loss: 2.27942 accuracy: 0.178\n",
      "Epoch  1, CIFAR-10 Batch 4:   loss: 2.27584 accuracy: 0.1902\n",
      "Epoch  1, CIFAR-10 Batch 5:   loss: 2.2796 accuracy: 0.1764\n",
      "Epoch  2, CIFAR-10 Batch 1:   loss: 2.29058 accuracy: 0.1626\n",
      "Epoch  2, CIFAR-10 Batch 2:   loss: 2.28532 accuracy: 0.1736\n",
      "Epoch  2, CIFAR-10 Batch 3:   loss: 2.278 accuracy: 0.1822\n",
      "Epoch  2, CIFAR-10 Batch 4:   loss: 2.25673 accuracy: 0.1808\n",
      "Epoch  2, CIFAR-10 Batch 5:   loss: 2.22409 accuracy: 0.1956\n",
      "Epoch  3, CIFAR-10 Batch 1:   loss: 2.20716 accuracy: 0.2018\n",
      "Epoch  3, CIFAR-10 Batch 2:   loss: 2.1478 accuracy: 0.2136\n",
      "Epoch  3, CIFAR-10 Batch 3:   loss: 2.13888 accuracy: 0.2278\n",
      "Epoch  3, CIFAR-10 Batch 4:   loss: 2.09141 accuracy: 0.2446\n",
      "Epoch  3, CIFAR-10 Batch 5:   loss: 2.07668 accuracy: 0.2534\n",
      "Epoch  4, CIFAR-10 Batch 1:   loss: 2.10284 accuracy: 0.2626\n",
      "Epoch  4, CIFAR-10 Batch 2:   loss: 2.04413 accuracy: 0.2626\n",
      "Epoch  4, CIFAR-10 Batch 3:   loss: 2.04098 accuracy: 0.2778\n",
      "Epoch  4, CIFAR-10 Batch 4:   loss: 2.00262 accuracy: 0.2932\n",
      "Epoch  4, CIFAR-10 Batch 5:   loss: 1.9976 accuracy: 0.2858\n",
      "Epoch  5, CIFAR-10 Batch 1:   loss: 2.03413 accuracy: 0.2868\n",
      "Epoch  5, CIFAR-10 Batch 2:   loss: 1.97098 accuracy: 0.311\n",
      "Epoch  5, CIFAR-10 Batch 3:   loss: 1.96227 accuracy: 0.3192\n",
      "Epoch  5, CIFAR-10 Batch 4:   loss: 1.92921 accuracy: 0.3204\n",
      "Epoch  5, CIFAR-10 Batch 5:   loss: 1.93285 accuracy: 0.3164\n",
      "Epoch  6, CIFAR-10 Batch 1:   loss: 1.96972 accuracy: 0.3248\n",
      "Epoch  6, CIFAR-10 Batch 2:   loss: 1.90752 accuracy: 0.337\n",
      "Epoch  6, CIFAR-10 Batch 3:   loss: 1.88468 accuracy: 0.338\n",
      "Epoch  6, CIFAR-10 Batch 4:   loss: 1.85794 accuracy: 0.3342\n",
      "Epoch  6, CIFAR-10 Batch 5:   loss: 1.87049 accuracy: 0.3308\n",
      "Epoch  7, CIFAR-10 Batch 1:   loss: 1.92312 accuracy: 0.3448\n",
      "Epoch  7, CIFAR-10 Batch 2:   loss: 1.86444 accuracy: 0.3482\n",
      "Epoch  7, CIFAR-10 Batch 3:   loss: 1.81033 accuracy: 0.357\n",
      "Epoch  7, CIFAR-10 Batch 4:   loss: 1.81058 accuracy: 0.3522\n",
      "Epoch  7, CIFAR-10 Batch 5:   loss: 1.83656 accuracy: 0.3482\n",
      "Epoch  8, CIFAR-10 Batch 1:   loss: 1.88922 accuracy: 0.3628\n",
      "Epoch  8, CIFAR-10 Batch 2:   loss: 1.8176 accuracy: 0.3642\n",
      "Epoch  8, CIFAR-10 Batch 3:   loss: 1.76652 accuracy: 0.3728\n",
      "Epoch  8, CIFAR-10 Batch 4:   loss: 1.77151 accuracy: 0.3656\n",
      "Epoch  8, CIFAR-10 Batch 5:   loss: 1.79463 accuracy: 0.3556\n",
      "Epoch  9, CIFAR-10 Batch 1:   loss: 1.86234 accuracy: 0.362\n",
      "Epoch  9, CIFAR-10 Batch 2:   loss: 1.79664 accuracy: 0.3718\n",
      "Epoch  9, CIFAR-10 Batch 3:   loss: 1.72941 accuracy: 0.3758\n",
      "Epoch  9, CIFAR-10 Batch 4:   loss: 1.73362 accuracy: 0.3792\n",
      "Epoch  9, CIFAR-10 Batch 5:   loss: 1.78144 accuracy: 0.3576\n",
      "Epoch 10, CIFAR-10 Batch 1:   loss: 1.82337 accuracy: 0.3762\n",
      "Epoch 10, CIFAR-10 Batch 2:   loss: 1.7912 accuracy: 0.38\n",
      "Epoch 10, CIFAR-10 Batch 3:   loss: 1.67754 accuracy: 0.392\n",
      "Epoch 10, CIFAR-10 Batch 4:   loss: 1.70735 accuracy: 0.3808\n",
      "Epoch 10, CIFAR-10 Batch 5:   loss: 1.74505 accuracy: 0.3712\n",
      "Epoch 11, CIFAR-10 Batch 1:   loss: 1.80254 accuracy: 0.3928\n",
      "Epoch 11, CIFAR-10 Batch 2:   loss: 1.74372 accuracy: 0.3886\n",
      "Epoch 11, CIFAR-10 Batch 3:   loss: 1.62831 accuracy: 0.3998\n",
      "Epoch 11, CIFAR-10 Batch 4:   loss: 1.66996 accuracy: 0.3938\n",
      "Epoch 11, CIFAR-10 Batch 5:   loss: 1.70717 accuracy: 0.3826\n",
      "Epoch 12, CIFAR-10 Batch 1:   loss: 1.76541 accuracy: 0.4006\n",
      "Epoch 12, CIFAR-10 Batch 2:   loss: 1.72484 accuracy: 0.3978\n",
      "Epoch 12, CIFAR-10 Batch 3:   loss: 1.58673 accuracy: 0.419\n",
      "Epoch 12, CIFAR-10 Batch 4:   loss: 1.65501 accuracy: 0.4094\n",
      "Epoch 12, CIFAR-10 Batch 5:   loss: 1.68647 accuracy: 0.3988\n",
      "Epoch 13, CIFAR-10 Batch 1:   loss: 1.72166 accuracy: 0.404\n",
      "Epoch 13, CIFAR-10 Batch 2:   loss: 1.73457 accuracy: 0.3978\n",
      "Epoch 13, CIFAR-10 Batch 3:   loss: 1.55824 accuracy: 0.4102\n",
      "Epoch 13, CIFAR-10 Batch 4:   loss: 1.62104 accuracy: 0.416\n",
      "Epoch 13, CIFAR-10 Batch 5:   loss: 1.6443 accuracy: 0.4162\n",
      "Epoch 14, CIFAR-10 Batch 1:   loss: 1.69323 accuracy: 0.4212\n",
      "Epoch 14, CIFAR-10 Batch 2:   loss: 1.63577 accuracy: 0.4214\n",
      "Epoch 14, CIFAR-10 Batch 3:   loss: 1.50719 accuracy: 0.4376\n",
      "Epoch 14, CIFAR-10 Batch 4:   loss: 1.59394 accuracy: 0.4232\n",
      "Epoch 14, CIFAR-10 Batch 5:   loss: 1.60992 accuracy: 0.4334\n",
      "Epoch 15, CIFAR-10 Batch 1:   loss: 1.68526 accuracy: 0.4246\n",
      "Epoch 15, CIFAR-10 Batch 2:   loss: 1.62651 accuracy: 0.4382\n",
      "Epoch 15, CIFAR-10 Batch 3:   loss: 1.48382 accuracy: 0.4424\n",
      "Epoch 15, CIFAR-10 Batch 4:   loss: 1.55872 accuracy: 0.4394\n",
      "Epoch 15, CIFAR-10 Batch 5:   loss: 1.57379 accuracy: 0.4344\n",
      "Epoch 16, CIFAR-10 Batch 1:   loss: 1.64214 accuracy: 0.4366\n",
      "Epoch 16, CIFAR-10 Batch 2:   loss: 1.61412 accuracy: 0.4438\n",
      "Epoch 16, CIFAR-10 Batch 3:   loss: 1.45009 accuracy: 0.4502\n",
      "Epoch 16, CIFAR-10 Batch 4:   loss: 1.55725 accuracy: 0.4438\n",
      "Epoch 16, CIFAR-10 Batch 5:   loss: 1.53716 accuracy: 0.45\n",
      "Epoch 17, CIFAR-10 Batch 1:   loss: 1.63303 accuracy: 0.4496\n",
      "Epoch 17, CIFAR-10 Batch 2:   loss: 1.59745 accuracy: 0.4448\n",
      "Epoch 17, CIFAR-10 Batch 3:   loss: 1.42171 accuracy: 0.4652\n",
      "Epoch 17, CIFAR-10 Batch 4:   loss: 1.52103 accuracy: 0.4474\n",
      "Epoch 17, CIFAR-10 Batch 5:   loss: 1.5358 accuracy: 0.4464\n",
      "Epoch 18, CIFAR-10 Batch 1:   loss: 1.59927 accuracy: 0.4542\n",
      "Epoch 18, CIFAR-10 Batch 2:   loss: 1.53321 accuracy: 0.4714\n",
      "Epoch 18, CIFAR-10 Batch 3:   loss: 1.38488 accuracy: 0.469\n",
      "Epoch 18, CIFAR-10 Batch 4:   loss: 1.50117 accuracy: 0.4552\n",
      "Epoch 18, CIFAR-10 Batch 5:   loss: 1.49201 accuracy: 0.4648\n",
      "Epoch 19, CIFAR-10 Batch 1:   loss: 1.55613 accuracy: 0.4644\n",
      "Epoch 19, CIFAR-10 Batch 2:   loss: 1.58268 accuracy: 0.4576\n",
      "Epoch 19, CIFAR-10 Batch 3:   loss: 1.36031 accuracy: 0.484\n",
      "Epoch 19, CIFAR-10 Batch 4:   loss: 1.46251 accuracy: 0.4698\n",
      "Epoch 19, CIFAR-10 Batch 5:   loss: 1.45684 accuracy: 0.4812\n",
      "Epoch 20, CIFAR-10 Batch 1:   loss: 1.57743 accuracy: 0.4642\n",
      "Epoch 20, CIFAR-10 Batch 2:   loss: 1.48885 accuracy: 0.4838\n",
      "Epoch 20, CIFAR-10 Batch 3:   loss: 1.35076 accuracy: 0.4878\n",
      "Epoch 20, CIFAR-10 Batch 4:   loss: 1.41878 accuracy: 0.4824\n",
      "Epoch 20, CIFAR-10 Batch 5:   loss: 1.4448 accuracy: 0.4818\n",
      "Epoch 21, CIFAR-10 Batch 1:   loss: 1.51631 accuracy: 0.4824\n",
      "Epoch 21, CIFAR-10 Batch 2:   loss: 1.46073 accuracy: 0.4966\n",
      "Epoch 21, CIFAR-10 Batch 3:   loss: 1.32558 accuracy: 0.4926\n",
      "Epoch 21, CIFAR-10 Batch 4:   loss: 1.3849 accuracy: 0.4936\n",
      "Epoch 21, CIFAR-10 Batch 5:   loss: 1.44109 accuracy: 0.4788\n",
      "Epoch 22, CIFAR-10 Batch 1:   loss: 1.48818 accuracy: 0.5032\n",
      "Epoch 22, CIFAR-10 Batch 2:   loss: 1.52889 accuracy: 0.4676\n",
      "Epoch 22, CIFAR-10 Batch 3:   loss: 1.31241 accuracy: 0.491\n",
      "Epoch 22, CIFAR-10 Batch 4:   loss: 1.39186 accuracy: 0.4898\n",
      "Epoch 22, CIFAR-10 Batch 5:   loss: 1.38311 accuracy: 0.4984\n",
      "Epoch 23, CIFAR-10 Batch 1:   loss: 1.47633 accuracy: 0.4978\n",
      "Epoch 23, CIFAR-10 Batch 2:   loss: 1.46618 accuracy: 0.4944\n",
      "Epoch 23, CIFAR-10 Batch 3:   loss: 1.29045 accuracy: 0.504\n",
      "Epoch 23, CIFAR-10 Batch 4:   loss: 1.34881 accuracy: 0.506\n",
      "Epoch 23, CIFAR-10 Batch 5:   loss: 1.3932 accuracy: 0.4948\n",
      "Epoch 24, CIFAR-10 Batch 1:   loss: 1.44386 accuracy: 0.5032\n",
      "Epoch 24, CIFAR-10 Batch 2:   loss: 1.42473 accuracy: 0.508\n",
      "Epoch 24, CIFAR-10 Batch 3:   loss: 1.26388 accuracy: 0.5234\n",
      "Epoch 24, CIFAR-10 Batch 4:   loss: 1.34372 accuracy: 0.5104\n",
      "Epoch 24, CIFAR-10 Batch 5:   loss: 1.36519 accuracy: 0.5046\n",
      "Epoch 25, CIFAR-10 Batch 1:   loss: 1.44534 accuracy: 0.5082\n",
      "Epoch 25, CIFAR-10 Batch 2:   loss: 1.37726 accuracy: 0.5146\n",
      "Epoch 25, CIFAR-10 Batch 3:   loss: 1.26721 accuracy: 0.5164\n",
      "Epoch 25, CIFAR-10 Batch 4:   loss: 1.31642 accuracy: 0.5156\n",
      "Epoch 25, CIFAR-10 Batch 5:   loss: 1.32332 accuracy: 0.5234\n",
      "Epoch 26, CIFAR-10 Batch 1:   loss: 1.4161 accuracy: 0.5164\n",
      "Epoch 26, CIFAR-10 Batch 2:   loss: 1.37637 accuracy: 0.5244\n",
      "Epoch 26, CIFAR-10 Batch 3:   loss: 1.24296 accuracy: 0.5218\n",
      "Epoch 26, CIFAR-10 Batch 4:   loss: 1.28958 accuracy: 0.5224\n",
      "Epoch 26, CIFAR-10 Batch 5:   loss: 1.34357 accuracy: 0.5076\n",
      "Epoch 27, CIFAR-10 Batch 1:   loss: 1.45804 accuracy: 0.5154\n",
      "Epoch 27, CIFAR-10 Batch 2:   loss: 1.33397 accuracy: 0.5316\n",
      "Epoch 27, CIFAR-10 Batch 3:   loss: 1.2196 accuracy: 0.5356\n",
      "Epoch 27, CIFAR-10 Batch 4:   loss: 1.29786 accuracy: 0.5232\n",
      "Epoch 27, CIFAR-10 Batch 5:   loss: 1.31828 accuracy: 0.5268\n",
      "Epoch 28, CIFAR-10 Batch 1:   loss: 1.41681 accuracy: 0.5232\n",
      "Epoch 28, CIFAR-10 Batch 2:   loss: 1.32077 accuracy: 0.5338\n",
      "Epoch 28, CIFAR-10 Batch 3:   loss: 1.23657 accuracy: 0.5272\n",
      "Epoch 28, CIFAR-10 Batch 4:   loss: 1.27015 accuracy: 0.5262\n",
      "Epoch 28, CIFAR-10 Batch 5:   loss: 1.29538 accuracy: 0.5362\n",
      "Epoch 29, CIFAR-10 Batch 1:   loss: 1.37511 accuracy: 0.535\n",
      "Epoch 29, CIFAR-10 Batch 2:   loss: 1.3452 accuracy: 0.5234\n",
      "Epoch 29, CIFAR-10 Batch 3:   loss: 1.20477 accuracy: 0.5406\n",
      "Epoch 29, CIFAR-10 Batch 4:   loss: 1.278 accuracy: 0.5294\n",
      "Epoch 29, CIFAR-10 Batch 5:   loss: 1.32275 accuracy: 0.523\n",
      "Epoch 30, CIFAR-10 Batch 1:   loss: 1.3817 accuracy: 0.529\n",
      "Epoch 30, CIFAR-10 Batch 2:   loss: 1.29664 accuracy: 0.5454\n",
      "Epoch 30, CIFAR-10 Batch 3:   loss: 1.21573 accuracy: 0.539\n",
      "Epoch 30, CIFAR-10 Batch 4:   loss: 1.25126 accuracy: 0.5306\n",
      "Epoch 30, CIFAR-10 Batch 5:   loss: 1.27887 accuracy: 0.5392\n",
      "Epoch 31, CIFAR-10 Batch 1:   loss: 1.37132 accuracy: 0.5408\n",
      "Epoch 31, CIFAR-10 Batch 2:   loss: 1.29068 accuracy: 0.5434\n",
      "Epoch 31, CIFAR-10 Batch 3:   loss: 1.1932 accuracy: 0.545\n",
      "Epoch 31, CIFAR-10 Batch 4:   loss: 1.24813 accuracy: 0.5454\n",
      "Epoch 31, CIFAR-10 Batch 5:   loss: 1.29954 accuracy: 0.5266\n",
      "Epoch 32, CIFAR-10 Batch 1:   loss: 1.36016 accuracy: 0.5404\n",
      "Epoch 32, CIFAR-10 Batch 2:   loss: 1.28532 accuracy: 0.5486\n",
      "Epoch 32, CIFAR-10 Batch 3:   loss: 1.191 accuracy: 0.5312\n",
      "Epoch 32, CIFAR-10 Batch 4:   loss: 1.22592 accuracy: 0.5448\n",
      "Epoch 32, CIFAR-10 Batch 5:   loss: 1.29286 accuracy: 0.5346\n",
      "Epoch 33, CIFAR-10 Batch 1:   loss: 1.34572 accuracy: 0.5424\n",
      "Epoch 33, CIFAR-10 Batch 2:   loss: 1.3154 accuracy: 0.5386\n",
      "Epoch 33, CIFAR-10 Batch 3:   loss: 1.16475 accuracy: 0.5538\n",
      "Epoch 33, CIFAR-10 Batch 4:   loss: 1.20698 accuracy: 0.55\n",
      "Epoch 33, CIFAR-10 Batch 5:   loss: 1.24499 accuracy: 0.5436\n",
      "Epoch 34, CIFAR-10 Batch 1:   loss: 1.35949 accuracy: 0.5398\n",
      "Epoch 34, CIFAR-10 Batch 2:   loss: 1.31534 accuracy: 0.5498\n",
      "Epoch 34, CIFAR-10 Batch 3:   loss: 1.13257 accuracy: 0.5546\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 34, CIFAR-10 Batch 4:   loss: 1.17129 accuracy: 0.5578\n",
      "Epoch 34, CIFAR-10 Batch 5:   loss: 1.23731 accuracy: 0.5464\n",
      "Epoch 35, CIFAR-10 Batch 1:   loss: 1.34099 accuracy: 0.544\n",
      "Epoch 35, CIFAR-10 Batch 2:   loss: 1.29646 accuracy: 0.5452\n",
      "Epoch 35, CIFAR-10 Batch 3:   loss: 1.12836 accuracy: 0.5712\n",
      "Epoch 35, CIFAR-10 Batch 4:   loss: 1.1927 accuracy: 0.5474\n",
      "Epoch 35, CIFAR-10 Batch 5:   loss: 1.23805 accuracy: 0.5496\n",
      "Epoch 36, CIFAR-10 Batch 1:   loss: 1.3371 accuracy: 0.5436\n",
      "Epoch 36, CIFAR-10 Batch 2:   loss: 1.24515 accuracy: 0.5676\n",
      "Epoch 36, CIFAR-10 Batch 3:   loss: 1.13838 accuracy: 0.5596\n",
      "Epoch 36, CIFAR-10 Batch 4:   loss: 1.14898 accuracy: 0.5678\n",
      "Epoch 36, CIFAR-10 Batch 5:   loss: 1.20897 accuracy: 0.5614\n",
      "Epoch 37, CIFAR-10 Batch 1:   loss: 1.32545 accuracy: 0.5546\n",
      "Epoch 37, CIFAR-10 Batch 2:   loss: 1.22407 accuracy: 0.5622\n",
      "Epoch 37, CIFAR-10 Batch 3:   loss: 1.1365 accuracy: 0.5586\n",
      "Epoch 37, CIFAR-10 Batch 4:   loss: 1.15816 accuracy: 0.5608\n",
      "Epoch 37, CIFAR-10 Batch 5:   loss: 1.20626 accuracy: 0.5604\n",
      "Epoch 38, CIFAR-10 Batch 1:   loss: 1.29789 accuracy: 0.5568\n",
      "Epoch 38, CIFAR-10 Batch 2:   loss: 1.23172 accuracy: 0.5624\n",
      "Epoch 38, CIFAR-10 Batch 3:   loss: 1.11236 accuracy: 0.5634\n",
      "Epoch 38, CIFAR-10 Batch 4:   loss: 1.14188 accuracy: 0.5674\n",
      "Epoch 38, CIFAR-10 Batch 5:   loss: 1.20541 accuracy: 0.5624\n",
      "Epoch 39, CIFAR-10 Batch 1:   loss: 1.24357 accuracy: 0.5712\n",
      "Epoch 39, CIFAR-10 Batch 2:   loss: 1.21595 accuracy: 0.5744\n",
      "Epoch 39, CIFAR-10 Batch 3:   loss: 1.0982 accuracy: 0.5806\n",
      "Epoch 39, CIFAR-10 Batch 4:   loss: 1.14922 accuracy: 0.568\n",
      "Epoch 39, CIFAR-10 Batch 5:   loss: 1.17161 accuracy: 0.5658\n",
      "Epoch 40, CIFAR-10 Batch 1:   loss: 1.26599 accuracy: 0.5756\n",
      "Epoch 40, CIFAR-10 Batch 2:   loss: 1.23195 accuracy: 0.5678\n",
      "Epoch 40, CIFAR-10 Batch 3:   loss: 1.07601 accuracy: 0.5952\n",
      "Epoch 40, CIFAR-10 Batch 4:   loss: 1.12633 accuracy: 0.568\n",
      "Epoch 40, CIFAR-10 Batch 5:   loss: 1.20746 accuracy: 0.56\n",
      "Epoch 41, CIFAR-10 Batch 1:   loss: 1.26584 accuracy: 0.5774\n",
      "Epoch 41, CIFAR-10 Batch 2:   loss: 1.20396 accuracy: 0.5772\n",
      "Epoch 41, CIFAR-10 Batch 3:   loss: 1.089 accuracy: 0.585\n",
      "Epoch 41, CIFAR-10 Batch 4:   loss: 1.10712 accuracy: 0.5832\n",
      "Epoch 41, CIFAR-10 Batch 5:   loss: 1.15597 accuracy: 0.5774\n",
      "Epoch 42, CIFAR-10 Batch 1:   loss: 1.25047 accuracy: 0.5882\n",
      "Epoch 42, CIFAR-10 Batch 2:   loss: 1.20281 accuracy: 0.5746\n",
      "Epoch 42, CIFAR-10 Batch 3:   loss: 1.0497 accuracy: 0.5938\n",
      "Epoch 42, CIFAR-10 Batch 4:   loss: 1.08719 accuracy: 0.5892\n",
      "Epoch 42, CIFAR-10 Batch 5:   loss: 1.18201 accuracy: 0.565\n",
      "Epoch 43, CIFAR-10 Batch 1:   loss: 1.23007 accuracy: 0.586\n",
      "Epoch 43, CIFAR-10 Batch 2:   loss: 1.2022 accuracy: 0.569\n",
      "Epoch 43, CIFAR-10 Batch 3:   loss: 1.05039 accuracy: 0.5936\n",
      "Epoch 43, CIFAR-10 Batch 4:   loss: 1.09703 accuracy: 0.5776\n",
      "Epoch 43, CIFAR-10 Batch 5:   loss: 1.17299 accuracy: 0.565\n",
      "Epoch 44, CIFAR-10 Batch 1:   loss: 1.22482 accuracy: 0.5936\n",
      "Epoch 44, CIFAR-10 Batch 2:   loss: 1.26827 accuracy: 0.5516\n",
      "Epoch 44, CIFAR-10 Batch 3:   loss: 1.04463 accuracy: 0.6032\n",
      "Epoch 44, CIFAR-10 Batch 4:   loss: 1.09002 accuracy: 0.5832\n",
      "Epoch 44, CIFAR-10 Batch 5:   loss: 1.18866 accuracy: 0.5598\n",
      "Epoch 45, CIFAR-10 Batch 1:   loss: 1.18268 accuracy: 0.5968\n",
      "Epoch 45, CIFAR-10 Batch 2:   loss: 1.20474 accuracy: 0.5728\n",
      "Epoch 45, CIFAR-10 Batch 3:   loss: 1.02949 accuracy: 0.5946\n",
      "Epoch 45, CIFAR-10 Batch 4:   loss: 1.082 accuracy: 0.597\n",
      "Epoch 45, CIFAR-10 Batch 5:   loss: 1.11679 accuracy: 0.5884\n",
      "Epoch 46, CIFAR-10 Batch 1:   loss: 1.2238 accuracy: 0.5906\n",
      "Epoch 46, CIFAR-10 Batch 2:   loss: 1.24956 accuracy: 0.566\n",
      "Epoch 46, CIFAR-10 Batch 3:   loss: 1.03615 accuracy: 0.5988\n",
      "Epoch 46, CIFAR-10 Batch 4:   loss: 1.05648 accuracy: 0.592\n",
      "Epoch 46, CIFAR-10 Batch 5:   loss: 1.15725 accuracy: 0.5752\n",
      "Epoch 47, CIFAR-10 Batch 1:   loss: 1.21883 accuracy: 0.5812\n",
      "Epoch 47, CIFAR-10 Batch 2:   loss: 1.19631 accuracy: 0.5728\n",
      "Epoch 47, CIFAR-10 Batch 3:   loss: 1.0203 accuracy: 0.602\n",
      "Epoch 47, CIFAR-10 Batch 4:   loss: 1.0582 accuracy: 0.5934\n",
      "Epoch 47, CIFAR-10 Batch 5:   loss: 1.14182 accuracy: 0.5766\n",
      "Epoch 48, CIFAR-10 Batch 1:   loss: 1.18289 accuracy: 0.5994\n",
      "Epoch 48, CIFAR-10 Batch 2:   loss: 1.21937 accuracy: 0.5702\n",
      "Epoch 48, CIFAR-10 Batch 3:   loss: 1.02184 accuracy: 0.6076\n",
      "Epoch 48, CIFAR-10 Batch 4:   loss: 1.04412 accuracy: 0.5948\n",
      "Epoch 48, CIFAR-10 Batch 5:   loss: 1.09552 accuracy: 0.588\n",
      "Epoch 49, CIFAR-10 Batch 1:   loss: 1.18319 accuracy: 0.5998\n",
      "Epoch 49, CIFAR-10 Batch 2:   loss: 1.1638 accuracy: 0.584\n",
      "Epoch 49, CIFAR-10 Batch 3:   loss: 1.06082 accuracy: 0.5764\n",
      "Epoch 49, CIFAR-10 Batch 4:   loss: 1.03809 accuracy: 0.5982\n",
      "Epoch 49, CIFAR-10 Batch 5:   loss: 1.10784 accuracy: 0.59\n",
      "Epoch 50, CIFAR-10 Batch 1:   loss: 1.161 accuracy: 0.6038\n",
      "Epoch 50, CIFAR-10 Batch 2:   loss: 1.14021 accuracy: 0.598\n",
      "Epoch 50, CIFAR-10 Batch 3:   loss: 1.04043 accuracy: 0.5872\n",
      "Epoch 50, CIFAR-10 Batch 4:   loss: 1.01352 accuracy: 0.614\n",
      "Epoch 50, CIFAR-10 Batch 5:   loss: 1.08172 accuracy: 0.598\n",
      "Epoch 51, CIFAR-10 Batch 1:   loss: 1.20273 accuracy: 0.5976\n",
      "Epoch 51, CIFAR-10 Batch 2:   loss: 1.1496 accuracy: 0.5858\n",
      "Epoch 51, CIFAR-10 Batch 3:   loss: 1.01596 accuracy: 0.5962\n",
      "Epoch 51, CIFAR-10 Batch 4:   loss: 1.02108 accuracy: 0.607\n",
      "Epoch 51, CIFAR-10 Batch 5:   loss: 1.06085 accuracy: 0.6084\n",
      "Epoch 52, CIFAR-10 Batch 1:   loss: 1.14244 accuracy: 0.6118\n",
      "Epoch 52, CIFAR-10 Batch 2:   loss: 1.15703 accuracy: 0.5846\n",
      "Epoch 52, CIFAR-10 Batch 3:   loss: 1.01902 accuracy: 0.6022\n",
      "Epoch 52, CIFAR-10 Batch 4:   loss: 1.02632 accuracy: 0.6044\n",
      "Epoch 52, CIFAR-10 Batch 5:   loss: 1.06664 accuracy: 0.6028\n",
      "Epoch 53, CIFAR-10 Batch 1:   loss: 1.1555 accuracy: 0.6038\n",
      "Epoch 53, CIFAR-10 Batch 2:   loss: 1.10327 accuracy: 0.6096\n",
      "Epoch 53, CIFAR-10 Batch 3:   loss: 1.00624 accuracy: 0.6028\n",
      "Epoch 53, CIFAR-10 Batch 4:   loss: 1.01797 accuracy: 0.601\n",
      "Epoch 53, CIFAR-10 Batch 5:   loss: 1.10065 accuracy: 0.586\n",
      "Epoch 54, CIFAR-10 Batch 1:   loss: 1.15347 accuracy: 0.6076\n",
      "Epoch 54, CIFAR-10 Batch 2:   loss: 1.1191 accuracy: 0.5958\n",
      "Epoch 54, CIFAR-10 Batch 3:   loss: 0.979843 accuracy: 0.6084\n",
      "Epoch 54, CIFAR-10 Batch 4:   loss: 0.994201 accuracy: 0.6228\n",
      "Epoch 54, CIFAR-10 Batch 5:   loss: 1.07758 accuracy: 0.6008\n",
      "Epoch 55, CIFAR-10 Batch 1:   loss: 1.13533 accuracy: 0.6212\n",
      "Epoch 55, CIFAR-10 Batch 2:   loss: 1.1283 accuracy: 0.5988\n",
      "Epoch 55, CIFAR-10 Batch 3:   loss: 0.99719 accuracy: 0.6036\n",
      "Epoch 55, CIFAR-10 Batch 4:   loss: 0.985166 accuracy: 0.621\n",
      "Epoch 55, CIFAR-10 Batch 5:   loss: 1.06567 accuracy: 0.5992\n",
      "Epoch 56, CIFAR-10 Batch 1:   loss: 1.14122 accuracy: 0.6128\n",
      "Epoch 56, CIFAR-10 Batch 2:   loss: 1.07626 accuracy: 0.613\n",
      "Epoch 56, CIFAR-10 Batch 3:   loss: 0.954736 accuracy: 0.6234\n",
      "Epoch 56, CIFAR-10 Batch 4:   loss: 0.992174 accuracy: 0.6136\n",
      "Epoch 56, CIFAR-10 Batch 5:   loss: 1.05937 accuracy: 0.6086\n",
      "Epoch 57, CIFAR-10 Batch 1:   loss: 1.13149 accuracy: 0.6166\n",
      "Epoch 57, CIFAR-10 Batch 2:   loss: 1.08045 accuracy: 0.6158\n",
      "Epoch 57, CIFAR-10 Batch 3:   loss: 0.959136 accuracy: 0.6168\n",
      "Epoch 57, CIFAR-10 Batch 4:   loss: 0.96614 accuracy: 0.6274\n",
      "Epoch 57, CIFAR-10 Batch 5:   loss: 1.07761 accuracy: 0.6004\n",
      "Epoch 58, CIFAR-10 Batch 1:   loss: 1.12749 accuracy: 0.6158\n",
      "Epoch 58, CIFAR-10 Batch 2:   loss: 1.07673 accuracy: 0.6128\n",
      "Epoch 58, CIFAR-10 Batch 3:   loss: 0.951256 accuracy: 0.6188\n",
      "Epoch 58, CIFAR-10 Batch 4:   loss: 0.972749 accuracy: 0.6236\n",
      "Epoch 58, CIFAR-10 Batch 5:   loss: 1.04511 accuracy: 0.61\n",
      "Epoch 59, CIFAR-10 Batch 1:   loss: 1.14849 accuracy: 0.6174\n",
      "Epoch 59, CIFAR-10 Batch 2:   loss: 1.03158 accuracy: 0.626\n",
      "Epoch 59, CIFAR-10 Batch 3:   loss: 0.952313 accuracy: 0.6164\n",
      "Epoch 59, CIFAR-10 Batch 4:   loss: 0.967531 accuracy: 0.6216\n",
      "Epoch 59, CIFAR-10 Batch 5:   loss: 0.987434 accuracy: 0.6256\n",
      "Epoch 60, CIFAR-10 Batch 1:   loss: 1.08516 accuracy: 0.637\n",
      "Epoch 60, CIFAR-10 Batch 2:   loss: 1.0434 accuracy: 0.6226\n",
      "Epoch 60, CIFAR-10 Batch 3:   loss: 0.928855 accuracy: 0.6344\n",
      "Epoch 60, CIFAR-10 Batch 4:   loss: 0.951745 accuracy: 0.6296\n",
      "Epoch 60, CIFAR-10 Batch 5:   loss: 1.00475 accuracy: 0.619\n",
      "Epoch 61, CIFAR-10 Batch 1:   loss: 1.10221 accuracy: 0.6304\n",
      "Epoch 61, CIFAR-10 Batch 2:   loss: 1.05309 accuracy: 0.6182\n",
      "Epoch 61, CIFAR-10 Batch 3:   loss: 0.93784 accuracy: 0.6324\n",
      "Epoch 61, CIFAR-10 Batch 4:   loss: 0.931843 accuracy: 0.6328\n",
      "Epoch 61, CIFAR-10 Batch 5:   loss: 1.03231 accuracy: 0.6118\n",
      "Epoch 62, CIFAR-10 Batch 1:   loss: 1.07641 accuracy: 0.6364\n",
      "Epoch 62, CIFAR-10 Batch 2:   loss: 1.05888 accuracy: 0.6222\n",
      "Epoch 62, CIFAR-10 Batch 3:   loss: 0.92974 accuracy: 0.627\n",
      "Epoch 62, CIFAR-10 Batch 4:   loss: 0.956798 accuracy: 0.6248\n",
      "Epoch 62, CIFAR-10 Batch 5:   loss: 1.01705 accuracy: 0.6166\n",
      "Epoch 63, CIFAR-10 Batch 1:   loss: 1.06348 accuracy: 0.646\n",
      "Epoch 63, CIFAR-10 Batch 2:   loss: 1.05801 accuracy: 0.614\n",
      "Epoch 63, CIFAR-10 Batch 3:   loss: 0.961438 accuracy: 0.6196\n",
      "Epoch 63, CIFAR-10 Batch 4:   loss: 0.977675 accuracy: 0.6196\n",
      "Epoch 63, CIFAR-10 Batch 5:   loss: 0.995599 accuracy: 0.6232\n",
      "Epoch 64, CIFAR-10 Batch 1:   loss: 1.07827 accuracy: 0.6412\n",
      "Epoch 64, CIFAR-10 Batch 2:   loss: 1.0378 accuracy: 0.6228\n",
      "Epoch 64, CIFAR-10 Batch 3:   loss: 0.913243 accuracy: 0.6378\n",
      "Epoch 64, CIFAR-10 Batch 4:   loss: 0.926884 accuracy: 0.6338\n",
      "Epoch 64, CIFAR-10 Batch 5:   loss: 0.994729 accuracy: 0.6172\n",
      "Epoch 65, CIFAR-10 Batch 1:   loss: 1.06559 accuracy: 0.6444\n",
      "Epoch 65, CIFAR-10 Batch 2:   loss: 1.04181 accuracy: 0.6222\n",
      "Epoch 65, CIFAR-10 Batch 3:   loss: 0.91854 accuracy: 0.6348\n",
      "Epoch 65, CIFAR-10 Batch 4:   loss: 0.929064 accuracy: 0.6368\n",
      "Epoch 65, CIFAR-10 Batch 5:   loss: 0.997967 accuracy: 0.6196\n",
      "Epoch 66, CIFAR-10 Batch 1:   loss: 1.05406 accuracy: 0.6416\n",
      "Epoch 66, CIFAR-10 Batch 2:   loss: 1.03251 accuracy: 0.6262\n",
      "Epoch 66, CIFAR-10 Batch 3:   loss: 0.919417 accuracy: 0.6336\n",
      "Epoch 66, CIFAR-10 Batch 4:   loss: 0.927493 accuracy: 0.6436\n",
      "Epoch 66, CIFAR-10 Batch 5:   loss: 1.04182 accuracy: 0.6042\n",
      "Epoch 67, CIFAR-10 Batch 1:   loss: 1.02918 accuracy: 0.646\n",
      "Epoch 67, CIFAR-10 Batch 2:   loss: 1.0446 accuracy: 0.6252\n",
      "Epoch 67, CIFAR-10 Batch 3:   loss: 0.910338 accuracy: 0.6352\n",
      "Epoch 67, CIFAR-10 Batch 4:   loss: 0.915163 accuracy: 0.6434\n",
      "Epoch 67, CIFAR-10 Batch 5:   loss: 0.992162 accuracy: 0.623\n",
      "Epoch 68, CIFAR-10 Batch 1:   loss: 1.04472 accuracy: 0.6448\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 68, CIFAR-10 Batch 2:   loss: 1.00114 accuracy: 0.6322\n",
      "Epoch 68, CIFAR-10 Batch 3:   loss: 0.902961 accuracy: 0.635\n",
      "Epoch 68, CIFAR-10 Batch 4:   loss: 0.914366 accuracy: 0.641\n",
      "Epoch 68, CIFAR-10 Batch 5:   loss: 0.95421 accuracy: 0.6406\n",
      "Epoch 69, CIFAR-10 Batch 1:   loss: 1.02927 accuracy: 0.6518\n",
      "Epoch 69, CIFAR-10 Batch 2:   loss: 0.991409 accuracy: 0.6466\n",
      "Epoch 69, CIFAR-10 Batch 3:   loss: 0.87708 accuracy: 0.6508\n",
      "Epoch 69, CIFAR-10 Batch 4:   loss: 0.932293 accuracy: 0.6396\n",
      "Epoch 69, CIFAR-10 Batch 5:   loss: 0.960598 accuracy: 0.6348\n",
      "Epoch 70, CIFAR-10 Batch 1:   loss: 1.02651 accuracy: 0.651\n",
      "Epoch 70, CIFAR-10 Batch 2:   loss: 1.03264 accuracy: 0.6222\n",
      "Epoch 70, CIFAR-10 Batch 3:   loss: 0.884435 accuracy: 0.6432\n",
      "Epoch 70, CIFAR-10 Batch 4:   loss: 0.944496 accuracy: 0.6264\n",
      "Epoch 70, CIFAR-10 Batch 5:   loss: 0.978 accuracy: 0.6284\n",
      "Epoch 71, CIFAR-10 Batch 1:   loss: 1.01876 accuracy: 0.6492\n",
      "Epoch 71, CIFAR-10 Batch 2:   loss: 0.993271 accuracy: 0.6368\n",
      "Epoch 71, CIFAR-10 Batch 3:   loss: 0.915106 accuracy: 0.6302\n",
      "Epoch 71, CIFAR-10 Batch 4:   loss: 0.902649 accuracy: 0.6512\n",
      "Epoch 71, CIFAR-10 Batch 5:   loss: 0.935426 accuracy: 0.6466\n",
      "Epoch 72, CIFAR-10 Batch 1:   loss: 1.01806 accuracy: 0.6492\n",
      "Epoch 72, CIFAR-10 Batch 2:   loss: 0.998399 accuracy: 0.637\n",
      "Epoch 72, CIFAR-10 Batch 3:   loss: 0.910749 accuracy: 0.6346\n",
      "Epoch 72, CIFAR-10 Batch 4:   loss: 0.889622 accuracy: 0.6538\n",
      "Epoch 72, CIFAR-10 Batch 5:   loss: 0.954381 accuracy: 0.6374\n",
      "Epoch 73, CIFAR-10 Batch 1:   loss: 1.00015 accuracy: 0.6552\n",
      "Epoch 73, CIFAR-10 Batch 2:   loss: 0.973373 accuracy: 0.6376\n",
      "Epoch 73, CIFAR-10 Batch 3:   loss: 0.869279 accuracy: 0.6422\n",
      "Epoch 73, CIFAR-10 Batch 4:   loss: 0.910354 accuracy: 0.6456\n",
      "Epoch 73, CIFAR-10 Batch 5:   loss: 0.96299 accuracy: 0.6294\n",
      "Epoch 74, CIFAR-10 Batch 1:   loss: 1.02236 accuracy: 0.659\n",
      "Epoch 74, CIFAR-10 Batch 2:   loss: 0.999849 accuracy: 0.628\n",
      "Epoch 74, CIFAR-10 Batch 3:   loss: 0.865583 accuracy: 0.6462\n",
      "Epoch 74, CIFAR-10 Batch 4:   loss: 0.889763 accuracy: 0.647\n",
      "Epoch 74, CIFAR-10 Batch 5:   loss: 0.985295 accuracy: 0.6244\n",
      "Epoch 75, CIFAR-10 Batch 1:   loss: 1.00944 accuracy: 0.654\n",
      "Epoch 75, CIFAR-10 Batch 2:   loss: 0.953682 accuracy: 0.649\n",
      "Epoch 75, CIFAR-10 Batch 3:   loss: 0.868534 accuracy: 0.6486\n",
      "Epoch 75, CIFAR-10 Batch 4:   loss: 0.903898 accuracy: 0.6438\n",
      "Epoch 75, CIFAR-10 Batch 5:   loss: 0.962364 accuracy: 0.6296\n",
      "Epoch 76, CIFAR-10 Batch 1:   loss: 1.02381 accuracy: 0.6608\n",
      "Epoch 76, CIFAR-10 Batch 2:   loss: 0.977085 accuracy: 0.6396\n",
      "Epoch 76, CIFAR-10 Batch 3:   loss: 0.877669 accuracy: 0.6386\n",
      "Epoch 76, CIFAR-10 Batch 4:   loss: 0.89318 accuracy: 0.6476\n",
      "Epoch 76, CIFAR-10 Batch 5:   loss: 0.992085 accuracy: 0.6226\n",
      "Epoch 77, CIFAR-10 Batch 1:   loss: 0.964281 accuracy: 0.6614\n",
      "Epoch 77, CIFAR-10 Batch 2:   loss: 0.957205 accuracy: 0.6472\n",
      "Epoch 77, CIFAR-10 Batch 3:   loss: 0.854171 accuracy: 0.6568\n",
      "Epoch 77, CIFAR-10 Batch 4:   loss: 0.892522 accuracy: 0.6456\n",
      "Epoch 77, CIFAR-10 Batch 5:   loss: 0.942064 accuracy: 0.6376\n",
      "Epoch 78, CIFAR-10 Batch 1:   loss: 1.00675 accuracy: 0.666\n",
      "Epoch 78, CIFAR-10 Batch 2:   loss: 0.970601 accuracy: 0.638\n",
      "Epoch 78, CIFAR-10 Batch 3:   loss: 0.88142 accuracy: 0.6376\n",
      "Epoch 78, CIFAR-10 Batch 4:   loss: 0.863148 accuracy: 0.6586\n",
      "Epoch 78, CIFAR-10 Batch 5:   loss: 0.980809 accuracy: 0.6298\n",
      "Epoch 79, CIFAR-10 Batch 1:   loss: 1.01143 accuracy: 0.6622\n",
      "Epoch 79, CIFAR-10 Batch 2:   loss: 0.944739 accuracy: 0.649\n",
      "Epoch 79, CIFAR-10 Batch 3:   loss: 0.861261 accuracy: 0.6502\n",
      "Epoch 79, CIFAR-10 Batch 4:   loss: 0.869005 accuracy: 0.6658\n",
      "Epoch 79, CIFAR-10 Batch 5:   loss: 0.927216 accuracy: 0.6468\n",
      "Epoch 80, CIFAR-10 Batch 1:   loss: 0.976267 accuracy: 0.6696\n",
      "Epoch 80, CIFAR-10 Batch 2:   loss: 0.919858 accuracy: 0.6584\n",
      "Epoch 80, CIFAR-10 Batch 3:   loss: 0.863351 accuracy: 0.642\n",
      "Epoch 80, CIFAR-10 Batch 4:   loss: 0.860893 accuracy: 0.6652\n",
      "Epoch 80, CIFAR-10 Batch 5:   loss: 0.92788 accuracy: 0.641\n",
      "Epoch 81, CIFAR-10 Batch 1:   loss: 1.00542 accuracy: 0.6582\n",
      "Epoch 81, CIFAR-10 Batch 2:   loss: 1.01721 accuracy: 0.6242\n",
      "Epoch 81, CIFAR-10 Batch 3:   loss: 0.858554 accuracy: 0.6466\n",
      "Epoch 81, CIFAR-10 Batch 4:   loss: 0.848936 accuracy: 0.6618\n",
      "Epoch 81, CIFAR-10 Batch 5:   loss: 0.957819 accuracy: 0.6286\n",
      "Epoch 82, CIFAR-10 Batch 1:   loss: 0.984002 accuracy: 0.6624\n",
      "Epoch 82, CIFAR-10 Batch 2:   loss: 0.916205 accuracy: 0.6678\n",
      "Epoch 82, CIFAR-10 Batch 3:   loss: 0.833957 accuracy: 0.6576\n",
      "Epoch 82, CIFAR-10 Batch 4:   loss: 0.881697 accuracy: 0.6524\n",
      "Epoch 82, CIFAR-10 Batch 5:   loss: 0.940829 accuracy: 0.635\n",
      "Epoch 83, CIFAR-10 Batch 1:   loss: 0.978455 accuracy: 0.669\n",
      "Epoch 83, CIFAR-10 Batch 2:   loss: 0.903228 accuracy: 0.6606\n",
      "Epoch 83, CIFAR-10 Batch 3:   loss: 0.854184 accuracy: 0.658\n",
      "Epoch 83, CIFAR-10 Batch 4:   loss: 0.862937 accuracy: 0.655\n",
      "Epoch 83, CIFAR-10 Batch 5:   loss: 0.951119 accuracy: 0.6388\n",
      "Epoch 84, CIFAR-10 Batch 1:   loss: 0.978833 accuracy: 0.6658\n",
      "Epoch 84, CIFAR-10 Batch 2:   loss: 0.889179 accuracy: 0.6694\n",
      "Epoch 84, CIFAR-10 Batch 3:   loss: 0.844755 accuracy: 0.652\n",
      "Epoch 84, CIFAR-10 Batch 4:   loss: 0.844329 accuracy: 0.6742\n",
      "Epoch 84, CIFAR-10 Batch 5:   loss: 0.887196 accuracy: 0.6632\n",
      "Epoch 85, CIFAR-10 Batch 1:   loss: 0.996229 accuracy: 0.66\n",
      "Epoch 85, CIFAR-10 Batch 2:   loss: 0.92517 accuracy: 0.661\n",
      "Epoch 85, CIFAR-10 Batch 3:   loss: 0.837957 accuracy: 0.6602\n",
      "Epoch 85, CIFAR-10 Batch 4:   loss: 0.837901 accuracy: 0.6668\n",
      "Epoch 85, CIFAR-10 Batch 5:   loss: 0.911828 accuracy: 0.6502\n",
      "Epoch 86, CIFAR-10 Batch 1:   loss: 0.980834 accuracy: 0.6622\n",
      "Epoch 86, CIFAR-10 Batch 2:   loss: 0.913607 accuracy: 0.6538\n",
      "Epoch 86, CIFAR-10 Batch 3:   loss: 0.814526 accuracy: 0.666\n",
      "Epoch 86, CIFAR-10 Batch 4:   loss: 0.828211 accuracy: 0.6726\n",
      "Epoch 86, CIFAR-10 Batch 5:   loss: 0.934211 accuracy: 0.6424\n",
      "Epoch 87, CIFAR-10 Batch 1:   loss: 0.926857 accuracy: 0.6702\n",
      "Epoch 87, CIFAR-10 Batch 2:   loss: 0.873256 accuracy: 0.6782\n",
      "Epoch 87, CIFAR-10 Batch 3:   loss: 0.818579 accuracy: 0.6624\n",
      "Epoch 87, CIFAR-10 Batch 4:   loss: 0.840638 accuracy: 0.6658\n",
      "Epoch 87, CIFAR-10 Batch 5:   loss: 0.902523 accuracy: 0.6472\n",
      "Epoch 88, CIFAR-10 Batch 1:   loss: 0.974227 accuracy: 0.6702\n",
      "Epoch 88, CIFAR-10 Batch 2:   loss: 0.880562 accuracy: 0.6728\n",
      "Epoch 88, CIFAR-10 Batch 3:   loss: 0.822177 accuracy: 0.6592\n",
      "Epoch 88, CIFAR-10 Batch 4:   loss: 0.82436 accuracy: 0.6728\n",
      "Epoch 88, CIFAR-10 Batch 5:   loss: 0.872372 accuracy: 0.6586\n",
      "Epoch 89, CIFAR-10 Batch 1:   loss: 0.958395 accuracy: 0.671\n",
      "Epoch 89, CIFAR-10 Batch 2:   loss: 0.879147 accuracy: 0.6682\n",
      "Epoch 89, CIFAR-10 Batch 3:   loss: 0.827586 accuracy: 0.6614\n",
      "Epoch 89, CIFAR-10 Batch 4:   loss: 0.836876 accuracy: 0.6664\n",
      "Epoch 89, CIFAR-10 Batch 5:   loss: 0.887691 accuracy: 0.659\n",
      "Epoch 90, CIFAR-10 Batch 1:   loss: 0.9575 accuracy: 0.6756\n",
      "Epoch 90, CIFAR-10 Batch 2:   loss: 0.893292 accuracy: 0.664\n",
      "Epoch 90, CIFAR-10 Batch 3:   loss: 0.826196 accuracy: 0.6572\n",
      "Epoch 90, CIFAR-10 Batch 4:   loss: 0.825764 accuracy: 0.6732\n",
      "Epoch 90, CIFAR-10 Batch 5:   loss: 0.85599 accuracy: 0.6658\n",
      "Epoch 91, CIFAR-10 Batch 1:   loss: 0.939902 accuracy: 0.6788\n",
      "Epoch 91, CIFAR-10 Batch 2:   loss: 0.89942 accuracy: 0.661\n",
      "Epoch 91, CIFAR-10 Batch 3:   loss: 0.821386 accuracy: 0.6608\n",
      "Epoch 91, CIFAR-10 Batch 4:   loss: 0.816536 accuracy: 0.6754\n",
      "Epoch 91, CIFAR-10 Batch 5:   loss: 0.864213 accuracy: 0.6596\n",
      "Epoch 92, CIFAR-10 Batch 1:   loss: 0.936813 accuracy: 0.6782\n",
      "Epoch 92, CIFAR-10 Batch 2:   loss: 0.90481 accuracy: 0.6656\n",
      "Epoch 92, CIFAR-10 Batch 3:   loss: 0.842155 accuracy: 0.6496\n",
      "Epoch 92, CIFAR-10 Batch 4:   loss: 0.817913 accuracy: 0.666\n",
      "Epoch 92, CIFAR-10 Batch 5:   loss: 0.858273 accuracy: 0.6618\n",
      "Epoch 93, CIFAR-10 Batch 1:   loss: 0.946542 accuracy: 0.6712\n",
      "Epoch 93, CIFAR-10 Batch 2:   loss: 0.908048 accuracy: 0.658\n",
      "Epoch 93, CIFAR-10 Batch 3:   loss: 0.803824 accuracy: 0.664\n",
      "Epoch 93, CIFAR-10 Batch 4:   loss: 0.830013 accuracy: 0.6678\n",
      "Epoch 93, CIFAR-10 Batch 5:   loss: 0.868867 accuracy: 0.6626\n",
      "Epoch 94, CIFAR-10 Batch 1:   loss: 0.913346 accuracy: 0.6772\n",
      "Epoch 94, CIFAR-10 Batch 2:   loss: 0.878247 accuracy: 0.6744\n",
      "Epoch 94, CIFAR-10 Batch 3:   loss: 0.804399 accuracy: 0.6654\n",
      "Epoch 94, CIFAR-10 Batch 4:   loss: 0.808151 accuracy: 0.6726\n",
      "Epoch 94, CIFAR-10 Batch 5:   loss: 0.865153 accuracy: 0.6592\n",
      "Epoch 95, CIFAR-10 Batch 1:   loss: 0.936469 accuracy: 0.6788\n",
      "Epoch 95, CIFAR-10 Batch 2:   loss: 0.904877 accuracy: 0.6562\n",
      "Epoch 95, CIFAR-10 Batch 3:   loss: 0.821694 accuracy: 0.6554\n",
      "Epoch 95, CIFAR-10 Batch 4:   loss: 0.788718 accuracy: 0.683\n",
      "Epoch 95, CIFAR-10 Batch 5:   loss: 0.887126 accuracy: 0.6516\n",
      "Epoch 96, CIFAR-10 Batch 1:   loss: 0.929794 accuracy: 0.6738\n",
      "Epoch 96, CIFAR-10 Batch 2:   loss: 0.894579 accuracy: 0.6628\n",
      "Epoch 96, CIFAR-10 Batch 3:   loss: 0.846743 accuracy: 0.6538\n",
      "Epoch 96, CIFAR-10 Batch 4:   loss: 0.816984 accuracy: 0.667\n",
      "Epoch 96, CIFAR-10 Batch 5:   loss: 0.84917 accuracy: 0.665\n",
      "Epoch 97, CIFAR-10 Batch 1:   loss: 0.96196 accuracy: 0.6796\n",
      "Epoch 97, CIFAR-10 Batch 2:   loss: 0.901887 accuracy: 0.6608\n",
      "Epoch 97, CIFAR-10 Batch 3:   loss: 0.773646 accuracy: 0.6736\n",
      "Epoch 97, CIFAR-10 Batch 4:   loss: 0.803705 accuracy: 0.669\n",
      "Epoch 97, CIFAR-10 Batch 5:   loss: 0.853136 accuracy: 0.6674\n",
      "Epoch 98, CIFAR-10 Batch 1:   loss: 0.927035 accuracy: 0.6826\n",
      "Epoch 98, CIFAR-10 Batch 2:   loss: 0.860041 accuracy: 0.675\n",
      "Epoch 98, CIFAR-10 Batch 3:   loss: 0.820921 accuracy: 0.6548\n",
      "Epoch 98, CIFAR-10 Batch 4:   loss: 0.787264 accuracy: 0.6768\n",
      "Epoch 98, CIFAR-10 Batch 5:   loss: 0.836326 accuracy: 0.6712\n",
      "Epoch 99, CIFAR-10 Batch 1:   loss: 0.913757 accuracy: 0.6806\n",
      "Epoch 99, CIFAR-10 Batch 2:   loss: 0.877576 accuracy: 0.6682\n",
      "Epoch 99, CIFAR-10 Batch 3:   loss: 0.760672 accuracy: 0.6772\n",
      "Epoch 99, CIFAR-10 Batch 4:   loss: 0.786479 accuracy: 0.6772\n",
      "Epoch 99, CIFAR-10 Batch 5:   loss: 0.851448 accuracy: 0.668\n",
      "Epoch 100, CIFAR-10 Batch 1:   loss: 0.909525 accuracy: 0.6832\n",
      "Epoch 100, CIFAR-10 Batch 2:   loss: 0.84537 accuracy: 0.6784\n",
      "Epoch 100, CIFAR-10 Batch 3:   loss: 0.780997 accuracy: 0.6752\n",
      "Epoch 100, CIFAR-10 Batch 4:   loss: 0.776299 accuracy: 0.6836\n",
      "Epoch 100, CIFAR-10 Batch 5:   loss: 0.847559 accuracy: 0.666\n",
      "Epoch 101, CIFAR-10 Batch 1:   loss: 0.963798 accuracy: 0.6798\n",
      "Epoch 101, CIFAR-10 Batch 2:   loss: 0.851401 accuracy: 0.6764\n",
      "Epoch 101, CIFAR-10 Batch 3:   loss: 0.780468 accuracy: 0.6754\n",
      "Epoch 101, CIFAR-10 Batch 4:   loss: 0.782445 accuracy: 0.6786\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 101, CIFAR-10 Batch 5:   loss: 0.832039 accuracy: 0.669\n",
      "Epoch 102, CIFAR-10 Batch 1:   loss: 0.900989 accuracy: 0.6866\n",
      "Epoch 102, CIFAR-10 Batch 2:   loss: 0.843305 accuracy: 0.685\n",
      "Epoch 102, CIFAR-10 Batch 3:   loss: 0.761183 accuracy: 0.6762\n",
      "Epoch 102, CIFAR-10 Batch 4:   loss: 0.77732 accuracy: 0.6818\n",
      "Epoch 102, CIFAR-10 Batch 5:   loss: 0.884062 accuracy: 0.6612\n",
      "Epoch 103, CIFAR-10 Batch 1:   loss: 0.89666 accuracy: 0.681\n",
      "Epoch 103, CIFAR-10 Batch 2:   loss: 0.855317 accuracy: 0.679\n",
      "Epoch 103, CIFAR-10 Batch 3:   loss: 0.791729 accuracy: 0.6652\n",
      "Epoch 103, CIFAR-10 Batch 4:   loss: 0.793539 accuracy: 0.6782\n",
      "Epoch 103, CIFAR-10 Batch 5:   loss: 0.846376 accuracy: 0.6668\n",
      "Epoch 104, CIFAR-10 Batch 1:   loss: 0.943565 accuracy: 0.6732\n",
      "Epoch 104, CIFAR-10 Batch 2:   loss: 0.867046 accuracy: 0.6708\n",
      "Epoch 104, CIFAR-10 Batch 3:   loss: 0.755519 accuracy: 0.6754\n",
      "Epoch 104, CIFAR-10 Batch 4:   loss: 0.757404 accuracy: 0.686\n",
      "Epoch 104, CIFAR-10 Batch 5:   loss: 0.825669 accuracy: 0.6716\n",
      "Epoch 105, CIFAR-10 Batch 1:   loss: 0.901398 accuracy: 0.6818\n",
      "Epoch 105, CIFAR-10 Batch 2:   loss: 0.836698 accuracy: 0.6766\n",
      "Epoch 105, CIFAR-10 Batch 3:   loss: 0.760213 accuracy: 0.6734\n",
      "Epoch 105, CIFAR-10 Batch 4:   loss: 0.783156 accuracy: 0.6828\n",
      "Epoch 105, CIFAR-10 Batch 5:   loss: 0.827936 accuracy: 0.6758\n",
      "Epoch 106, CIFAR-10 Batch 1:   loss: 0.919819 accuracy: 0.6752\n",
      "Epoch 106, CIFAR-10 Batch 2:   loss: 0.840105 accuracy: 0.679\n",
      "Epoch 106, CIFAR-10 Batch 3:   loss: 0.790575 accuracy: 0.6586\n",
      "Epoch 106, CIFAR-10 Batch 4:   loss: 0.762663 accuracy: 0.6866\n",
      "Epoch 106, CIFAR-10 Batch 5:   loss: 0.821186 accuracy: 0.6776\n",
      "Epoch 107, CIFAR-10 Batch 1:   loss: 0.886618 accuracy: 0.6902\n",
      "Epoch 107, CIFAR-10 Batch 2:   loss: 0.811065 accuracy: 0.6826\n",
      "Epoch 107, CIFAR-10 Batch 3:   loss: 0.798819 accuracy: 0.6654\n",
      "Epoch 107, CIFAR-10 Batch 4:   loss: 0.787584 accuracy: 0.686\n",
      "Epoch 107, CIFAR-10 Batch 5:   loss: 0.819822 accuracy: 0.6836\n",
      "Epoch 108, CIFAR-10 Batch 1:   loss: 0.884365 accuracy: 0.6894\n",
      "Epoch 108, CIFAR-10 Batch 2:   loss: 0.831865 accuracy: 0.6762\n",
      "Epoch 108, CIFAR-10 Batch 3:   loss: 0.750412 accuracy: 0.674\n",
      "Epoch 108, CIFAR-10 Batch 4:   loss: 0.755375 accuracy: 0.6842\n",
      "Epoch 108, CIFAR-10 Batch 5:   loss: 0.834314 accuracy: 0.6698\n",
      "Epoch 109, CIFAR-10 Batch 1:   loss: 0.917376 accuracy: 0.6848\n",
      "Epoch 109, CIFAR-10 Batch 2:   loss: 0.826437 accuracy: 0.6784\n",
      "Epoch 109, CIFAR-10 Batch 3:   loss: 0.771276 accuracy: 0.6712\n",
      "Epoch 109, CIFAR-10 Batch 4:   loss: 0.754437 accuracy: 0.6842\n",
      "Epoch 109, CIFAR-10 Batch 5:   loss: 0.793013 accuracy: 0.681\n",
      "Epoch 110, CIFAR-10 Batch 1:   loss: 0.864753 accuracy: 0.6924\n",
      "Epoch 110, CIFAR-10 Batch 2:   loss: 0.80909 accuracy: 0.6786\n",
      "Epoch 110, CIFAR-10 Batch 3:   loss: 0.795633 accuracy: 0.6576\n",
      "Epoch 110, CIFAR-10 Batch 4:   loss: 0.765101 accuracy: 0.6838\n",
      "Epoch 110, CIFAR-10 Batch 5:   loss: 0.796402 accuracy: 0.682\n",
      "Epoch 111, CIFAR-10 Batch 1:   loss: 0.872524 accuracy: 0.6864\n",
      "Epoch 111, CIFAR-10 Batch 2:   loss: 0.805434 accuracy: 0.6876\n",
      "Epoch 111, CIFAR-10 Batch 3:   loss: 0.738602 accuracy: 0.6804\n",
      "Epoch 111, CIFAR-10 Batch 4:   loss: 0.741752 accuracy: 0.6882\n",
      "Epoch 111, CIFAR-10 Batch 5:   loss: 0.79618 accuracy: 0.683\n",
      "Epoch 112, CIFAR-10 Batch 1:   loss: 0.907692 accuracy: 0.6812\n",
      "Epoch 112, CIFAR-10 Batch 2:   loss: 0.838109 accuracy: 0.675\n",
      "Epoch 112, CIFAR-10 Batch 3:   loss: 0.782694 accuracy: 0.6636\n",
      "Epoch 112, CIFAR-10 Batch 4:   loss: 0.748133 accuracy: 0.6858\n",
      "Epoch 112, CIFAR-10 Batch 5:   loss: 0.799845 accuracy: 0.6818\n",
      "Epoch 113, CIFAR-10 Batch 1:   loss: 0.866331 accuracy: 0.6866\n",
      "Epoch 113, CIFAR-10 Batch 2:   loss: 0.806978 accuracy: 0.6878\n",
      "Epoch 113, CIFAR-10 Batch 3:   loss: 0.745354 accuracy: 0.6792\n",
      "Epoch 113, CIFAR-10 Batch 4:   loss: 0.739581 accuracy: 0.6928\n",
      "Epoch 113, CIFAR-10 Batch 5:   loss: 0.824699 accuracy: 0.6752\n",
      "Epoch 114, CIFAR-10 Batch 1:   loss: 0.862157 accuracy: 0.6872\n",
      "Epoch 114, CIFAR-10 Batch 2:   loss: 0.825656 accuracy: 0.674\n",
      "Epoch 114, CIFAR-10 Batch 3:   loss: 0.727098 accuracy: 0.6804\n",
      "Epoch 114, CIFAR-10 Batch 4:   loss: 0.738322 accuracy: 0.6892\n",
      "Epoch 114, CIFAR-10 Batch 5:   loss: 0.781602 accuracy: 0.6864\n",
      "Epoch 115, CIFAR-10 Batch 1:   loss: 0.865254 accuracy: 0.6916\n",
      "Epoch 115, CIFAR-10 Batch 2:   loss: 0.785233 accuracy: 0.6888\n",
      "Epoch 115, CIFAR-10 Batch 3:   loss: 0.76751 accuracy: 0.673\n",
      "Epoch 115, CIFAR-10 Batch 4:   loss: 0.740378 accuracy: 0.6908\n",
      "Epoch 115, CIFAR-10 Batch 5:   loss: 0.80295 accuracy: 0.6794\n",
      "Epoch 116, CIFAR-10 Batch 1:   loss: 0.885092 accuracy: 0.6854\n",
      "Epoch 116, CIFAR-10 Batch 2:   loss: 0.84929 accuracy: 0.6768\n",
      "Epoch 116, CIFAR-10 Batch 3:   loss: 0.728886 accuracy: 0.6754\n",
      "Epoch 116, CIFAR-10 Batch 4:   loss: 0.744693 accuracy: 0.6902\n",
      "Epoch 116, CIFAR-10 Batch 5:   loss: 0.810399 accuracy: 0.675\n",
      "Epoch 117, CIFAR-10 Batch 1:   loss: 0.867103 accuracy: 0.6888\n",
      "Epoch 117, CIFAR-10 Batch 2:   loss: 0.796892 accuracy: 0.6866\n",
      "Epoch 117, CIFAR-10 Batch 3:   loss: 0.747544 accuracy: 0.6756\n",
      "Epoch 117, CIFAR-10 Batch 4:   loss: 0.765748 accuracy: 0.6802\n",
      "Epoch 117, CIFAR-10 Batch 5:   loss: 0.79927 accuracy: 0.682\n",
      "Epoch 118, CIFAR-10 Batch 1:   loss: 0.859221 accuracy: 0.6892\n",
      "Epoch 118, CIFAR-10 Batch 2:   loss: 0.824675 accuracy: 0.6842\n",
      "Epoch 118, CIFAR-10 Batch 3:   loss: 0.7331 accuracy: 0.6814\n",
      "Epoch 118, CIFAR-10 Batch 4:   loss: 0.742485 accuracy: 0.6882\n",
      "Epoch 118, CIFAR-10 Batch 5:   loss: 0.781481 accuracy: 0.6886\n",
      "Epoch 119, CIFAR-10 Batch 1:   loss: 0.840724 accuracy: 0.6938\n",
      "Epoch 119, CIFAR-10 Batch 2:   loss: 0.807993 accuracy: 0.6908\n",
      "Epoch 119, CIFAR-10 Batch 3:   loss: 0.764665 accuracy: 0.669\n",
      "Epoch 119, CIFAR-10 Batch 4:   loss: 0.725972 accuracy: 0.691\n",
      "Epoch 119, CIFAR-10 Batch 5:   loss: 0.822043 accuracy: 0.672\n",
      "Epoch 120, CIFAR-10 Batch 1:   loss: 0.872237 accuracy: 0.6934\n",
      "Epoch 120, CIFAR-10 Batch 2:   loss: 0.783448 accuracy: 0.6982\n",
      "Epoch 120, CIFAR-10 Batch 3:   loss: 0.70918 accuracy: 0.6918\n",
      "Epoch 120, CIFAR-10 Batch 4:   loss: 0.735868 accuracy: 0.6906\n",
      "Epoch 120, CIFAR-10 Batch 5:   loss: 0.806035 accuracy: 0.678\n",
      "Epoch 121, CIFAR-10 Batch 1:   loss: 0.884271 accuracy: 0.6856\n",
      "Epoch 121, CIFAR-10 Batch 2:   loss: 0.798079 accuracy: 0.6934\n",
      "Epoch 121, CIFAR-10 Batch 3:   loss: 0.719572 accuracy: 0.6812\n",
      "Epoch 121, CIFAR-10 Batch 4:   loss: 0.719761 accuracy: 0.6962\n",
      "Epoch 121, CIFAR-10 Batch 5:   loss: 0.824399 accuracy: 0.6682\n",
      "Epoch 122, CIFAR-10 Batch 1:   loss: 0.839132 accuracy: 0.6872\n",
      "Epoch 122, CIFAR-10 Batch 2:   loss: 0.791302 accuracy: 0.6898\n",
      "Epoch 122, CIFAR-10 Batch 3:   loss: 0.711425 accuracy: 0.686\n",
      "Epoch 122, CIFAR-10 Batch 4:   loss: 0.748249 accuracy: 0.6822\n",
      "Epoch 122, CIFAR-10 Batch 5:   loss: 0.829222 accuracy: 0.6734\n",
      "Epoch 123, CIFAR-10 Batch 1:   loss: 0.839549 accuracy: 0.6988\n",
      "Epoch 123, CIFAR-10 Batch 2:   loss: 0.783326 accuracy: 0.6892\n",
      "Epoch 123, CIFAR-10 Batch 3:   loss: 0.712955 accuracy: 0.691\n",
      "Epoch 123, CIFAR-10 Batch 4:   loss: 0.71451 accuracy: 0.6928\n",
      "Epoch 123, CIFAR-10 Batch 5:   loss: 0.752033 accuracy: 0.6946\n",
      "Epoch 124, CIFAR-10 Batch 1:   loss: 0.836711 accuracy: 0.7042\n",
      "Epoch 124, CIFAR-10 Batch 2:   loss: 0.805942 accuracy: 0.6914\n",
      "Epoch 124, CIFAR-10 Batch 3:   loss: 0.699906 accuracy: 0.6926\n",
      "Epoch 124, CIFAR-10 Batch 4:   loss: 0.721015 accuracy: 0.6896\n",
      "Epoch 124, CIFAR-10 Batch 5:   loss: 0.829893 accuracy: 0.6704\n",
      "Epoch 125, CIFAR-10 Batch 1:   loss: 0.85433 accuracy: 0.6982\n",
      "Epoch 125, CIFAR-10 Batch 2:   loss: 0.815381 accuracy: 0.6832\n",
      "Epoch 125, CIFAR-10 Batch 3:   loss: 0.7484 accuracy: 0.6738\n",
      "Epoch 125, CIFAR-10 Batch 4:   loss: 0.695801 accuracy: 0.7046\n",
      "Epoch 125, CIFAR-10 Batch 5:   loss: 0.814692 accuracy: 0.6748\n",
      "Epoch 126, CIFAR-10 Batch 1:   loss: 0.831685 accuracy: 0.6956\n",
      "Epoch 126, CIFAR-10 Batch 2:   loss: 0.806933 accuracy: 0.6842\n",
      "Epoch 126, CIFAR-10 Batch 3:   loss: 0.771737 accuracy: 0.6638\n",
      "Epoch 126, CIFAR-10 Batch 4:   loss: 0.730478 accuracy: 0.6842\n",
      "Epoch 126, CIFAR-10 Batch 5:   loss: 0.77547 accuracy: 0.6884\n",
      "Epoch 127, CIFAR-10 Batch 1:   loss: 0.828299 accuracy: 0.6988\n",
      "Epoch 127, CIFAR-10 Batch 2:   loss: 0.771354 accuracy: 0.7014\n",
      "Epoch 127, CIFAR-10 Batch 3:   loss: 0.690554 accuracy: 0.6942\n",
      "Epoch 127, CIFAR-10 Batch 4:   loss: 0.711778 accuracy: 0.6982\n",
      "Epoch 127, CIFAR-10 Batch 5:   loss: 0.771338 accuracy: 0.6852\n",
      "Epoch 128, CIFAR-10 Batch 1:   loss: 0.819265 accuracy: 0.7012\n",
      "Epoch 128, CIFAR-10 Batch 2:   loss: 0.766052 accuracy: 0.7004\n",
      "Epoch 128, CIFAR-10 Batch 3:   loss: 0.693761 accuracy: 0.69\n",
      "Epoch 128, CIFAR-10 Batch 4:   loss: 0.725262 accuracy: 0.7014\n",
      "Epoch 128, CIFAR-10 Batch 5:   loss: 0.751182 accuracy: 0.6944\n",
      "Epoch 129, CIFAR-10 Batch 1:   loss: 0.852679 accuracy: 0.6964\n",
      "Epoch 129, CIFAR-10 Batch 2:   loss: 0.78202 accuracy: 0.6896\n",
      "Epoch 129, CIFAR-10 Batch 3:   loss: 0.713655 accuracy: 0.6856\n",
      "Epoch 129, CIFAR-10 Batch 4:   loss: 0.733112 accuracy: 0.6878\n",
      "Epoch 129, CIFAR-10 Batch 5:   loss: 0.807914 accuracy: 0.6782\n",
      "Epoch 130, CIFAR-10 Batch 1:   loss: 0.847971 accuracy: 0.696\n",
      "Epoch 130, CIFAR-10 Batch 2:   loss: 0.749409 accuracy: 0.6962\n",
      "Epoch 130, CIFAR-10 Batch 3:   loss: 0.683871 accuracy: 0.6948\n",
      "Epoch 130, CIFAR-10 Batch 4:   loss: 0.708872 accuracy: 0.697\n",
      "Epoch 130, CIFAR-10 Batch 5:   loss: 0.767661 accuracy: 0.6872\n",
      "Epoch 131, CIFAR-10 Batch 1:   loss: 0.829286 accuracy: 0.7034\n",
      "Epoch 131, CIFAR-10 Batch 2:   loss: 0.76409 accuracy: 0.6948\n",
      "Epoch 131, CIFAR-10 Batch 3:   loss: 0.673577 accuracy: 0.6928\n",
      "Epoch 131, CIFAR-10 Batch 4:   loss: 0.706603 accuracy: 0.7048\n",
      "Epoch 131, CIFAR-10 Batch 5:   loss: 0.795578 accuracy: 0.6856\n",
      "Epoch 132, CIFAR-10 Batch 1:   loss: 0.858221 accuracy: 0.6894\n",
      "Epoch 132, CIFAR-10 Batch 2:   loss: 0.752363 accuracy: 0.6976\n",
      "Epoch 132, CIFAR-10 Batch 3:   loss: 0.693456 accuracy: 0.6892\n",
      "Epoch 132, CIFAR-10 Batch 4:   loss: 0.697396 accuracy: 0.6998\n",
      "Epoch 132, CIFAR-10 Batch 5:   loss: 0.757719 accuracy: 0.6928\n",
      "Epoch 133, CIFAR-10 Batch 1:   loss: 0.854961 accuracy: 0.6916\n",
      "Epoch 133, CIFAR-10 Batch 2:   loss: 0.76012 accuracy: 0.697\n",
      "Epoch 133, CIFAR-10 Batch 3:   loss: 0.699509 accuracy: 0.6858\n",
      "Epoch 133, CIFAR-10 Batch 4:   loss: 0.715877 accuracy: 0.691\n",
      "Epoch 133, CIFAR-10 Batch 5:   loss: 0.773899 accuracy: 0.6832\n",
      "Epoch 134, CIFAR-10 Batch 1:   loss: 0.811925 accuracy: 0.7016\n",
      "Epoch 134, CIFAR-10 Batch 2:   loss: 0.793145 accuracy: 0.69\n",
      "Epoch 134, CIFAR-10 Batch 3:   loss: 0.716233 accuracy: 0.679\n",
      "Epoch 134, CIFAR-10 Batch 4:   loss: 0.69856 accuracy: 0.7002\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 134, CIFAR-10 Batch 5:   loss: 0.776001 accuracy: 0.6826\n",
      "Epoch 135, CIFAR-10 Batch 1:   loss: 0.850925 accuracy: 0.691\n",
      "Epoch 135, CIFAR-10 Batch 2:   loss: 0.758287 accuracy: 0.6968\n",
      "Epoch 135, CIFAR-10 Batch 3:   loss: 0.693054 accuracy: 0.6892\n",
      "Epoch 135, CIFAR-10 Batch 4:   loss: 0.698438 accuracy: 0.702\n",
      "Epoch 135, CIFAR-10 Batch 5:   loss: 0.743676 accuracy: 0.6916\n",
      "Epoch 136, CIFAR-10 Batch 1:   loss: 0.812696 accuracy: 0.7008\n",
      "Epoch 136, CIFAR-10 Batch 2:   loss: 0.74598 accuracy: 0.6972\n",
      "Epoch 136, CIFAR-10 Batch 3:   loss: 0.67001 accuracy: 0.6888\n",
      "Epoch 136, CIFAR-10 Batch 4:   loss: 0.697675 accuracy: 0.7034\n",
      "Epoch 136, CIFAR-10 Batch 5:   loss: 0.757127 accuracy: 0.689\n",
      "Epoch 137, CIFAR-10 Batch 1:   loss: 0.799596 accuracy: 0.7042\n",
      "Epoch 137, CIFAR-10 Batch 2:   loss: 0.727028 accuracy: 0.7036\n",
      "Epoch 137, CIFAR-10 Batch 3:   loss: 0.677881 accuracy: 0.6874\n",
      "Epoch 137, CIFAR-10 Batch 4:   loss: 0.700708 accuracy: 0.7022\n",
      "Epoch 137, CIFAR-10 Batch 5:   loss: 0.765101 accuracy: 0.687\n",
      "Epoch 138, CIFAR-10 Batch 1:   loss: 0.81026 accuracy: 0.702\n",
      "Epoch 138, CIFAR-10 Batch 2:   loss: 0.770315 accuracy: 0.696\n",
      "Epoch 138, CIFAR-10 Batch 3:   loss: 0.684197 accuracy: 0.6906\n",
      "Epoch 138, CIFAR-10 Batch 4:   loss: 0.700112 accuracy: 0.6996\n",
      "Epoch 138, CIFAR-10 Batch 5:   loss: 0.728557 accuracy: 0.6958\n",
      "Epoch 139, CIFAR-10 Batch 1:   loss: 0.824361 accuracy: 0.7028\n",
      "Epoch 139, CIFAR-10 Batch 2:   loss: 0.749233 accuracy: 0.6998\n",
      "Epoch 139, CIFAR-10 Batch 3:   loss: 0.679272 accuracy: 0.695\n",
      "Epoch 139, CIFAR-10 Batch 4:   loss: 0.685254 accuracy: 0.7018\n",
      "Epoch 139, CIFAR-10 Batch 5:   loss: 0.720767 accuracy: 0.7024\n",
      "Epoch 140, CIFAR-10 Batch 1:   loss: 0.839688 accuracy: 0.697\n",
      "Epoch 140, CIFAR-10 Batch 2:   loss: 0.741873 accuracy: 0.7008\n",
      "Epoch 140, CIFAR-10 Batch 3:   loss: 0.663936 accuracy: 0.6916\n",
      "Epoch 140, CIFAR-10 Batch 4:   loss: 0.708045 accuracy: 0.6902\n",
      "Epoch 140, CIFAR-10 Batch 5:   loss: 0.738929 accuracy: 0.688\n",
      "Epoch 141, CIFAR-10 Batch 1:   loss: 0.795685 accuracy: 0.7048\n",
      "Epoch 141, CIFAR-10 Batch 2:   loss: 0.742225 accuracy: 0.7056\n",
      "Epoch 141, CIFAR-10 Batch 3:   loss: 0.680585 accuracy: 0.6866\n",
      "Epoch 141, CIFAR-10 Batch 4:   loss: 0.702437 accuracy: 0.6966\n",
      "Epoch 141, CIFAR-10 Batch 5:   loss: 0.746946 accuracy: 0.6836\n",
      "Epoch 142, CIFAR-10 Batch 1:   loss: 0.770938 accuracy: 0.7048\n",
      "Epoch 142, CIFAR-10 Batch 2:   loss: 0.747953 accuracy: 0.7036\n",
      "Epoch 142, CIFAR-10 Batch 3:   loss: 0.675185 accuracy: 0.6936\n",
      "Epoch 142, CIFAR-10 Batch 4:   loss: 0.701019 accuracy: 0.7054\n",
      "Epoch 142, CIFAR-10 Batch 5:   loss: 0.721717 accuracy: 0.6994\n",
      "Epoch 143, CIFAR-10 Batch 1:   loss: 0.813753 accuracy: 0.6956\n",
      "Epoch 143, CIFAR-10 Batch 2:   loss: 0.755487 accuracy: 0.695\n",
      "Epoch 143, CIFAR-10 Batch 3:   loss: 0.682744 accuracy: 0.6902\n",
      "Epoch 143, CIFAR-10 Batch 4:   loss: 0.685772 accuracy: 0.7038\n",
      "Epoch 143, CIFAR-10 Batch 5:   loss: 0.734746 accuracy: 0.6984\n",
      "Epoch 144, CIFAR-10 Batch 1:   loss: 0.797731 accuracy: 0.6964\n",
      "Epoch 144, CIFAR-10 Batch 2:   loss: 0.750243 accuracy: 0.6974\n",
      "Epoch 144, CIFAR-10 Batch 3:   loss: 0.678536 accuracy: 0.6956\n",
      "Epoch 144, CIFAR-10 Batch 4:   loss: 0.687985 accuracy: 0.7056\n",
      "Epoch 144, CIFAR-10 Batch 5:   loss: 0.739588 accuracy: 0.695\n",
      "Epoch 145, CIFAR-10 Batch 1:   loss: 0.808914 accuracy: 0.6978\n",
      "Epoch 145, CIFAR-10 Batch 2:   loss: 0.747691 accuracy: 0.7054\n",
      "Epoch 145, CIFAR-10 Batch 3:   loss: 0.661765 accuracy: 0.7006\n",
      "Epoch 145, CIFAR-10 Batch 4:   loss: 0.709933 accuracy: 0.6912\n",
      "Epoch 145, CIFAR-10 Batch 5:   loss: 0.731419 accuracy: 0.6942\n",
      "Epoch 146, CIFAR-10 Batch 1:   loss: 0.799616 accuracy: 0.7058\n",
      "Epoch 146, CIFAR-10 Batch 2:   loss: 0.741534 accuracy: 0.7048\n",
      "Epoch 146, CIFAR-10 Batch 3:   loss: 0.657936 accuracy: 0.7002\n",
      "Epoch 146, CIFAR-10 Batch 4:   loss: 0.710809 accuracy: 0.6976\n",
      "Epoch 146, CIFAR-10 Batch 5:   loss: 0.74946 accuracy: 0.6858\n",
      "Epoch 147, CIFAR-10 Batch 1:   loss: 0.765978 accuracy: 0.7088\n",
      "Epoch 147, CIFAR-10 Batch 2:   loss: 0.728067 accuracy: 0.704\n",
      "Epoch 147, CIFAR-10 Batch 3:   loss: 0.660849 accuracy: 0.7028\n",
      "Epoch 147, CIFAR-10 Batch 4:   loss: 0.686879 accuracy: 0.6998\n",
      "Epoch 147, CIFAR-10 Batch 5:   loss: 0.714439 accuracy: 0.6938\n",
      "Epoch 148, CIFAR-10 Batch 1:   loss: 0.751685 accuracy: 0.7058\n",
      "Epoch 148, CIFAR-10 Batch 2:   loss: 0.782649 accuracy: 0.692\n",
      "Epoch 148, CIFAR-10 Batch 3:   loss: 0.680624 accuracy: 0.6974\n",
      "Epoch 148, CIFAR-10 Batch 4:   loss: 0.683483 accuracy: 0.7052\n",
      "Epoch 148, CIFAR-10 Batch 5:   loss: 0.764405 accuracy: 0.6834\n",
      "Epoch 149, CIFAR-10 Batch 1:   loss: 0.788475 accuracy: 0.702\n",
      "Epoch 149, CIFAR-10 Batch 2:   loss: 0.751642 accuracy: 0.7022\n",
      "Epoch 149, CIFAR-10 Batch 3:   loss: 0.675338 accuracy: 0.7\n",
      "Epoch 149, CIFAR-10 Batch 4:   loss: 0.685753 accuracy: 0.7046\n",
      "Epoch 149, CIFAR-10 Batch 5:   loss: 0.734325 accuracy: 0.6978\n",
      "Epoch 150, CIFAR-10 Batch 1:   loss: 0.804573 accuracy: 0.7044\n",
      "Epoch 150, CIFAR-10 Batch 2:   loss: 0.743577 accuracy: 0.7032\n",
      "Epoch 150, CIFAR-10 Batch 3:   loss: 0.685783 accuracy: 0.6876\n",
      "Epoch 150, CIFAR-10 Batch 4:   loss: 0.668512 accuracy: 0.7006\n",
      "Epoch 150, CIFAR-10 Batch 5:   loss: 0.723203 accuracy: 0.6938\n",
      "Epoch 151, CIFAR-10 Batch 1:   loss: 0.778165 accuracy: 0.7054\n",
      "Epoch 151, CIFAR-10 Batch 2:   loss: 0.698348 accuracy: 0.7078\n",
      "Epoch 151, CIFAR-10 Batch 3:   loss: 0.645894 accuracy: 0.7042\n",
      "Epoch 151, CIFAR-10 Batch 4:   loss: 0.669235 accuracy: 0.7024\n",
      "Epoch 151, CIFAR-10 Batch 5:   loss: 0.737847 accuracy: 0.6934\n",
      "Epoch 152, CIFAR-10 Batch 1:   loss: 0.763887 accuracy: 0.7094\n",
      "Epoch 152, CIFAR-10 Batch 2:   loss: 0.71462 accuracy: 0.7124\n",
      "Epoch 152, CIFAR-10 Batch 3:   loss: 0.653206 accuracy: 0.7016\n",
      "Epoch 152, CIFAR-10 Batch 4:   loss: 0.654561 accuracy: 0.7058\n",
      "Epoch 152, CIFAR-10 Batch 5:   loss: 0.702048 accuracy: 0.7072\n",
      "Epoch 153, CIFAR-10 Batch 1:   loss: 0.765581 accuracy: 0.7132\n",
      "Epoch 153, CIFAR-10 Batch 2:   loss: 0.738167 accuracy: 0.701\n",
      "Epoch 153, CIFAR-10 Batch 3:   loss: 0.627271 accuracy: 0.7014\n",
      "Epoch 153, CIFAR-10 Batch 4:   loss: 0.659393 accuracy: 0.7066\n",
      "Epoch 153, CIFAR-10 Batch 5:   loss: 0.715581 accuracy: 0.6992\n",
      "Epoch 154, CIFAR-10 Batch 1:   loss: 0.798531 accuracy: 0.7038\n",
      "Epoch 154, CIFAR-10 Batch 2:   loss: 0.722669 accuracy: 0.7058\n",
      "Epoch 154, CIFAR-10 Batch 3:   loss: 0.6622 accuracy: 0.695\n",
      "Epoch 154, CIFAR-10 Batch 4:   loss: 0.647275 accuracy: 0.7076\n",
      "Epoch 154, CIFAR-10 Batch 5:   loss: 0.737722 accuracy: 0.6868\n",
      "Epoch 155, CIFAR-10 Batch 1:   loss: 0.76591 accuracy: 0.7112\n",
      "Epoch 155, CIFAR-10 Batch 2:   loss: 0.725173 accuracy: 0.707\n",
      "Epoch 155, CIFAR-10 Batch 3:   loss: 0.636355 accuracy: 0.6988\n",
      "Epoch 155, CIFAR-10 Batch 4:   loss: 0.650468 accuracy: 0.7096\n",
      "Epoch 155, CIFAR-10 Batch 5:   loss: 0.713532 accuracy: 0.7016\n",
      "Epoch 156, CIFAR-10 Batch 1:   loss: 0.776819 accuracy: 0.7078\n",
      "Epoch 156, CIFAR-10 Batch 2:   loss: 0.748527 accuracy: 0.7018\n",
      "Epoch 156, CIFAR-10 Batch 3:   loss: 0.627771 accuracy: 0.7008\n",
      "Epoch 156, CIFAR-10 Batch 4:   loss: 0.647332 accuracy: 0.7108\n",
      "Epoch 156, CIFAR-10 Batch 5:   loss: 0.725085 accuracy: 0.6946\n",
      "Epoch 157, CIFAR-10 Batch 1:   loss: 0.79378 accuracy: 0.705\n",
      "Epoch 157, CIFAR-10 Batch 2:   loss: 0.760013 accuracy: 0.6924\n",
      "Epoch 157, CIFAR-10 Batch 3:   loss: 0.654862 accuracy: 0.7038\n",
      "Epoch 157, CIFAR-10 Batch 4:   loss: 0.657214 accuracy: 0.7052\n",
      "Epoch 157, CIFAR-10 Batch 5:   loss: 0.719037 accuracy: 0.6962\n",
      "Epoch 158, CIFAR-10 Batch 1:   loss: 0.785757 accuracy: 0.7016\n",
      "Epoch 158, CIFAR-10 Batch 2:   loss: 0.741136 accuracy: 0.6988\n",
      "Epoch 158, CIFAR-10 Batch 3:   loss: 0.672715 accuracy: 0.6906\n",
      "Epoch 158, CIFAR-10 Batch 4:   loss: 0.655053 accuracy: 0.7104\n",
      "Epoch 158, CIFAR-10 Batch 5:   loss: 0.702252 accuracy: 0.7034\n",
      "Epoch 159, CIFAR-10 Batch 1:   loss: 0.76169 accuracy: 0.7078\n",
      "Epoch 159, CIFAR-10 Batch 2:   loss: 0.703488 accuracy: 0.7112\n",
      "Epoch 159, CIFAR-10 Batch 3:   loss: 0.647567 accuracy: 0.7062\n",
      "Epoch 159, CIFAR-10 Batch 4:   loss: 0.654374 accuracy: 0.709\n",
      "Epoch 159, CIFAR-10 Batch 5:   loss: 0.684561 accuracy: 0.7034\n",
      "Epoch 160, CIFAR-10 Batch 1:   loss: 0.742054 accuracy: 0.7136\n",
      "Epoch 160, CIFAR-10 Batch 2:   loss: 0.726075 accuracy: 0.7082\n",
      "Epoch 160, CIFAR-10 Batch 3:   loss: 0.696371 accuracy: 0.6792\n",
      "Epoch 160, CIFAR-10 Batch 4:   loss: 0.647866 accuracy: 0.7104\n",
      "Epoch 160, CIFAR-10 Batch 5:   loss: 0.701903 accuracy: 0.698\n",
      "Epoch 161, CIFAR-10 Batch 1:   loss: 0.757923 accuracy: 0.7102\n",
      "Epoch 161, CIFAR-10 Batch 2:   loss: 0.717698 accuracy: 0.708\n",
      "Epoch 161, CIFAR-10 Batch 3:   loss: 0.670653 accuracy: 0.6898\n",
      "Epoch 161, CIFAR-10 Batch 4:   loss: 0.646769 accuracy: 0.7058\n",
      "Epoch 161, CIFAR-10 Batch 5:   loss: 0.726929 accuracy: 0.6926\n",
      "Epoch 162, CIFAR-10 Batch 1:   loss: 0.756092 accuracy: 0.7058\n",
      "Epoch 162, CIFAR-10 Batch 2:   loss: 0.720624 accuracy: 0.712\n",
      "Epoch 162, CIFAR-10 Batch 3:   loss: 0.635455 accuracy: 0.7098\n",
      "Epoch 162, CIFAR-10 Batch 4:   loss: 0.649404 accuracy: 0.7132\n",
      "Epoch 162, CIFAR-10 Batch 5:   loss: 0.692015 accuracy: 0.7092\n",
      "Epoch 163, CIFAR-10 Batch 1:   loss: 0.766755 accuracy: 0.7118\n",
      "Epoch 163, CIFAR-10 Batch 2:   loss: 0.70316 accuracy: 0.7112\n",
      "Epoch 163, CIFAR-10 Batch 3:   loss: 0.628319 accuracy: 0.707\n",
      "Epoch 163, CIFAR-10 Batch 4:   loss: 0.63672 accuracy: 0.7146\n",
      "Epoch 163, CIFAR-10 Batch 5:   loss: 0.710583 accuracy: 0.7028\n",
      "Epoch 164, CIFAR-10 Batch 1:   loss: 0.761204 accuracy: 0.7118\n",
      "Epoch 164, CIFAR-10 Batch 2:   loss: 0.697816 accuracy: 0.7168\n",
      "Epoch 164, CIFAR-10 Batch 3:   loss: 0.642316 accuracy: 0.7026\n",
      "Epoch 164, CIFAR-10 Batch 4:   loss: 0.637609 accuracy: 0.7116\n",
      "Epoch 164, CIFAR-10 Batch 5:   loss: 0.707931 accuracy: 0.703\n",
      "Epoch 165, CIFAR-10 Batch 1:   loss: 0.732477 accuracy: 0.7144\n",
      "Epoch 165, CIFAR-10 Batch 2:   loss: 0.710088 accuracy: 0.7012\n",
      "Epoch 165, CIFAR-10 Batch 3:   loss: 0.626564 accuracy: 0.7028\n",
      "Epoch 165, CIFAR-10 Batch 4:   loss: 0.640821 accuracy: 0.709\n",
      "Epoch 165, CIFAR-10 Batch 5:   loss: 0.718321 accuracy: 0.6992\n",
      "Epoch 166, CIFAR-10 Batch 1:   loss: 0.746006 accuracy: 0.7096\n",
      "Epoch 166, CIFAR-10 Batch 2:   loss: 0.686839 accuracy: 0.714\n",
      "Epoch 166, CIFAR-10 Batch 3:   loss: 0.638737 accuracy: 0.7092\n",
      "Epoch 166, CIFAR-10 Batch 4:   loss: 0.627928 accuracy: 0.7176\n",
      "Epoch 166, CIFAR-10 Batch 5:   loss: 0.692114 accuracy: 0.7112\n",
      "Epoch 167, CIFAR-10 Batch 1:   loss: 0.753561 accuracy: 0.7148\n",
      "Epoch 167, CIFAR-10 Batch 2:   loss: 0.695634 accuracy: 0.7124\n",
      "Epoch 167, CIFAR-10 Batch 3:   loss: 0.652477 accuracy: 0.7012\n",
      "Epoch 167, CIFAR-10 Batch 4:   loss: 0.62866 accuracy: 0.7154\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 167, CIFAR-10 Batch 5:   loss: 0.675834 accuracy: 0.7124\n",
      "Epoch 168, CIFAR-10 Batch 1:   loss: 0.746157 accuracy: 0.7142\n",
      "Epoch 168, CIFAR-10 Batch 2:   loss: 0.685602 accuracy: 0.7108\n",
      "Epoch 168, CIFAR-10 Batch 3:   loss: 0.611515 accuracy: 0.7092\n",
      "Epoch 168, CIFAR-10 Batch 4:   loss: 0.637417 accuracy: 0.7138\n",
      "Epoch 168, CIFAR-10 Batch 5:   loss: 0.717544 accuracy: 0.697\n",
      "Epoch 169, CIFAR-10 Batch 1:   loss: 0.737372 accuracy: 0.7088\n",
      "Epoch 169, CIFAR-10 Batch 2:   loss: 0.719433 accuracy: 0.7044\n",
      "Epoch 169, CIFAR-10 Batch 3:   loss: 0.618569 accuracy: 0.7108\n",
      "Epoch 169, CIFAR-10 Batch 4:   loss: 0.63581 accuracy: 0.7116\n",
      "Epoch 169, CIFAR-10 Batch 5:   loss: 0.688354 accuracy: 0.7074\n",
      "Epoch 170, CIFAR-10 Batch 1:   loss: 0.77697 accuracy: 0.7148\n",
      "Epoch 170, CIFAR-10 Batch 2:   loss: 0.696238 accuracy: 0.7082\n",
      "Epoch 170, CIFAR-10 Batch 3:   loss: 0.64017 accuracy: 0.7006\n",
      "Epoch 170, CIFAR-10 Batch 4:   loss: 0.62576 accuracy: 0.7114\n",
      "Epoch 170, CIFAR-10 Batch 5:   loss: 0.680465 accuracy: 0.7062\n",
      "Epoch 171, CIFAR-10 Batch 1:   loss: 0.754498 accuracy: 0.7124\n",
      "Epoch 171, CIFAR-10 Batch 2:   loss: 0.700472 accuracy: 0.7124\n",
      "Epoch 171, CIFAR-10 Batch 3:   loss: 0.626971 accuracy: 0.7044\n",
      "Epoch 171, CIFAR-10 Batch 4:   loss: 0.629399 accuracy: 0.7118\n",
      "Epoch 171, CIFAR-10 Batch 5:   loss: 0.686838 accuracy: 0.7084\n",
      "Epoch 172, CIFAR-10 Batch 1:   loss: 0.754058 accuracy: 0.707\n",
      "Epoch 172, CIFAR-10 Batch 2:   loss: 0.676016 accuracy: 0.7132\n",
      "Epoch 172, CIFAR-10 Batch 3:   loss: 0.622509 accuracy: 0.7096\n",
      "Epoch 172, CIFAR-10 Batch 4:   loss: 0.610132 accuracy: 0.7142\n",
      "Epoch 172, CIFAR-10 Batch 5:   loss: 0.683829 accuracy: 0.708\n",
      "Epoch 173, CIFAR-10 Batch 1:   loss: 0.718309 accuracy: 0.719\n",
      "Epoch 173, CIFAR-10 Batch 2:   loss: 0.69182 accuracy: 0.7118\n",
      "Epoch 173, CIFAR-10 Batch 3:   loss: 0.620897 accuracy: 0.7036\n",
      "Epoch 173, CIFAR-10 Batch 4:   loss: 0.614806 accuracy: 0.7196\n",
      "Epoch 173, CIFAR-10 Batch 5:   loss: 0.658635 accuracy: 0.7154\n",
      "Epoch 174, CIFAR-10 Batch 1:   loss: 0.712145 accuracy: 0.7154\n",
      "Epoch 174, CIFAR-10 Batch 2:   loss: 0.674439 accuracy: 0.7132\n",
      "Epoch 174, CIFAR-10 Batch 3:   loss: 0.604366 accuracy: 0.7096\n",
      "Epoch 174, CIFAR-10 Batch 4:   loss: 0.645685 accuracy: 0.7082\n",
      "Epoch 174, CIFAR-10 Batch 5:   loss: 0.698221 accuracy: 0.7076\n",
      "Epoch 175, CIFAR-10 Batch 1:   loss: 0.725757 accuracy: 0.7138\n",
      "Epoch 175, CIFAR-10 Batch 2:   loss: 0.686585 accuracy: 0.7092\n",
      "Epoch 175, CIFAR-10 Batch 3:   loss: 0.608713 accuracy: 0.7142\n",
      "Epoch 175, CIFAR-10 Batch 4:   loss: 0.615806 accuracy: 0.7196\n",
      "Epoch 175, CIFAR-10 Batch 5:   loss: 0.69308 accuracy: 0.7126\n",
      "Epoch 176, CIFAR-10 Batch 1:   loss: 0.746346 accuracy: 0.7074\n",
      "Epoch 176, CIFAR-10 Batch 2:   loss: 0.672941 accuracy: 0.7136\n",
      "Epoch 176, CIFAR-10 Batch 3:   loss: 0.606784 accuracy: 0.7038\n",
      "Epoch 176, CIFAR-10 Batch 4:   loss: 0.621893 accuracy: 0.7172\n",
      "Epoch 176, CIFAR-10 Batch 5:   loss: 0.678007 accuracy: 0.7076\n",
      "Epoch 177, CIFAR-10 Batch 1:   loss: 0.714556 accuracy: 0.7116\n",
      "Epoch 177, CIFAR-10 Batch 2:   loss: 0.669384 accuracy: 0.7168\n",
      "Epoch 177, CIFAR-10 Batch 3:   loss: 0.598987 accuracy: 0.709\n",
      "Epoch 177, CIFAR-10 Batch 4:   loss: 0.621735 accuracy: 0.7114\n",
      "Epoch 177, CIFAR-10 Batch 5:   loss: 0.667481 accuracy: 0.7066\n",
      "Epoch 178, CIFAR-10 Batch 1:   loss: 0.736026 accuracy: 0.716\n",
      "Epoch 178, CIFAR-10 Batch 2:   loss: 0.701339 accuracy: 0.7048\n",
      "Epoch 178, CIFAR-10 Batch 3:   loss: 0.631204 accuracy: 0.6996\n",
      "Epoch 178, CIFAR-10 Batch 4:   loss: 0.625977 accuracy: 0.7118\n",
      "Epoch 178, CIFAR-10 Batch 5:   loss: 0.696379 accuracy: 0.7114\n",
      "Epoch 179, CIFAR-10 Batch 1:   loss: 0.728457 accuracy: 0.7124\n",
      "Epoch 179, CIFAR-10 Batch 2:   loss: 0.672059 accuracy: 0.7162\n",
      "Epoch 179, CIFAR-10 Batch 3:   loss: 0.627991 accuracy: 0.7036\n",
      "Epoch 179, CIFAR-10 Batch 4:   loss: 0.626459 accuracy: 0.7098\n",
      "Epoch 179, CIFAR-10 Batch 5:   loss: 0.673888 accuracy: 0.7088\n",
      "Epoch 180, CIFAR-10 Batch 1:   loss: 0.749209 accuracy: 0.7112\n",
      "Epoch 180, CIFAR-10 Batch 2:   loss: 0.674082 accuracy: 0.7068\n",
      "Epoch 180, CIFAR-10 Batch 3:   loss: 0.640075 accuracy: 0.7002\n",
      "Epoch 180, CIFAR-10 Batch 4:   loss: 0.621428 accuracy: 0.717\n",
      "Epoch 180, CIFAR-10 Batch 5:   loss: 0.694611 accuracy: 0.7042\n",
      "Epoch 181, CIFAR-10 Batch 1:   loss: 0.733245 accuracy: 0.7108\n",
      "Epoch 181, CIFAR-10 Batch 2:   loss: 0.683464 accuracy: 0.7092\n",
      "Epoch 181, CIFAR-10 Batch 3:   loss: 0.643543 accuracy: 0.6924\n",
      "Epoch 181, CIFAR-10 Batch 4:   loss: 0.610069 accuracy: 0.7156\n",
      "Epoch 181, CIFAR-10 Batch 5:   loss: 0.669673 accuracy: 0.7082\n",
      "Epoch 182, CIFAR-10 Batch 1:   loss: 0.695796 accuracy: 0.718\n",
      "Epoch 182, CIFAR-10 Batch 2:   loss: 0.658499 accuracy: 0.7142\n",
      "Epoch 182, CIFAR-10 Batch 3:   loss: 0.60088 accuracy: 0.7114\n",
      "Epoch 182, CIFAR-10 Batch 4:   loss: 0.63513 accuracy: 0.7096\n",
      "Epoch 182, CIFAR-10 Batch 5:   loss: 0.696757 accuracy: 0.6958\n",
      "Epoch 183, CIFAR-10 Batch 1:   loss: 0.744916 accuracy: 0.715\n",
      "Epoch 183, CIFAR-10 Batch 2:   loss: 0.667849 accuracy: 0.7158\n",
      "Epoch 183, CIFAR-10 Batch 3:   loss: 0.613606 accuracy: 0.7122\n",
      "Epoch 183, CIFAR-10 Batch 4:   loss: 0.60293 accuracy: 0.7146\n",
      "Epoch 183, CIFAR-10 Batch 5:   loss: 0.661671 accuracy: 0.7118\n",
      "Epoch 184, CIFAR-10 Batch 1:   loss: 0.717758 accuracy: 0.7146\n",
      "Epoch 184, CIFAR-10 Batch 2:   loss: 0.677339 accuracy: 0.7148\n",
      "Epoch 184, CIFAR-10 Batch 3:   loss: 0.609701 accuracy: 0.7102\n",
      "Epoch 184, CIFAR-10 Batch 4:   loss: 0.63035 accuracy: 0.7124\n",
      "Epoch 184, CIFAR-10 Batch 5:   loss: 0.658723 accuracy: 0.7152\n",
      "Epoch 185, CIFAR-10 Batch 1:   loss: 0.705722 accuracy: 0.7228\n",
      "Epoch 185, CIFAR-10 Batch 2:   loss: 0.66212 accuracy: 0.7218\n",
      "Epoch 185, CIFAR-10 Batch 3:   loss: 0.612681 accuracy: 0.7114\n",
      "Epoch 185, CIFAR-10 Batch 4:   loss: 0.606554 accuracy: 0.719\n",
      "Epoch 185, CIFAR-10 Batch 5:   loss: 0.652831 accuracy: 0.7116\n",
      "Epoch 186, CIFAR-10 Batch 1:   loss: 0.689671 accuracy: 0.7222\n",
      "Epoch 186, CIFAR-10 Batch 2:   loss: 0.656388 accuracy: 0.7144\n",
      "Epoch 186, CIFAR-10 Batch 3:   loss: 0.608671 accuracy: 0.717\n",
      "Epoch 186, CIFAR-10 Batch 4:   loss: 0.605635 accuracy: 0.725\n",
      "Epoch 186, CIFAR-10 Batch 5:   loss: 0.676264 accuracy: 0.7052\n",
      "Epoch 187, CIFAR-10 Batch 1:   loss: 0.719121 accuracy: 0.715\n",
      "Epoch 187, CIFAR-10 Batch 2:   loss: 0.693107 accuracy: 0.7096\n",
      "Epoch 187, CIFAR-10 Batch 3:   loss: 0.603371 accuracy: 0.7198\n",
      "Epoch 187, CIFAR-10 Batch 4:   loss: 0.597368 accuracy: 0.7206\n",
      "Epoch 187, CIFAR-10 Batch 5:   loss: 0.641054 accuracy: 0.7126\n",
      "Epoch 188, CIFAR-10 Batch 1:   loss: 0.703579 accuracy: 0.7178\n",
      "Epoch 188, CIFAR-10 Batch 2:   loss: 0.653908 accuracy: 0.721\n",
      "Epoch 188, CIFAR-10 Batch 3:   loss: 0.586926 accuracy: 0.72\n",
      "Epoch 188, CIFAR-10 Batch 4:   loss: 0.605908 accuracy: 0.7188\n",
      "Epoch 188, CIFAR-10 Batch 5:   loss: 0.645349 accuracy: 0.7142\n",
      "Epoch 189, CIFAR-10 Batch 1:   loss: 0.715971 accuracy: 0.7204\n",
      "Epoch 189, CIFAR-10 Batch 2:   loss: 0.693322 accuracy: 0.7028\n",
      "Epoch 189, CIFAR-10 Batch 3:   loss: 0.594618 accuracy: 0.7154\n",
      "Epoch 189, CIFAR-10 Batch 4:   loss: 0.608256 accuracy: 0.7186\n",
      "Epoch 189, CIFAR-10 Batch 5:   loss: 0.649319 accuracy: 0.7168\n",
      "Epoch 190, CIFAR-10 Batch 1:   loss: 0.716633 accuracy: 0.719\n",
      "Epoch 190, CIFAR-10 Batch 2:   loss: 0.661857 accuracy: 0.7134\n",
      "Epoch 190, CIFAR-10 Batch 3:   loss: 0.586349 accuracy: 0.7158\n",
      "Epoch 190, CIFAR-10 Batch 4:   loss: 0.600576 accuracy: 0.7156\n",
      "Epoch 190, CIFAR-10 Batch 5:   loss: 0.661203 accuracy: 0.7122\n",
      "Epoch 191, CIFAR-10 Batch 1:   loss: 0.754776 accuracy: 0.7084\n",
      "Epoch 191, CIFAR-10 Batch 2:   loss: 0.64729 accuracy: 0.715\n",
      "Epoch 191, CIFAR-10 Batch 3:   loss: 0.596187 accuracy: 0.7044\n",
      "Epoch 191, CIFAR-10 Batch 4:   loss: 0.584427 accuracy: 0.722\n",
      "Epoch 191, CIFAR-10 Batch 5:   loss: 0.641445 accuracy: 0.717\n",
      "Epoch 192, CIFAR-10 Batch 1:   loss: 0.701271 accuracy: 0.7214\n",
      "Epoch 192, CIFAR-10 Batch 2:   loss: 0.654107 accuracy: 0.7178\n",
      "Epoch 192, CIFAR-10 Batch 3:   loss: 0.583962 accuracy: 0.7194\n",
      "Epoch 192, CIFAR-10 Batch 4:   loss: 0.59358 accuracy: 0.7218\n",
      "Epoch 192, CIFAR-10 Batch 5:   loss: 0.640979 accuracy: 0.7128\n",
      "Epoch 193, CIFAR-10 Batch 1:   loss: 0.699825 accuracy: 0.7238\n",
      "Epoch 193, CIFAR-10 Batch 2:   loss: 0.6498 accuracy: 0.7208\n",
      "Epoch 193, CIFAR-10 Batch 3:   loss: 0.601093 accuracy: 0.7174\n",
      "Epoch 193, CIFAR-10 Batch 4:   loss: 0.594697 accuracy: 0.7234\n",
      "Epoch 193, CIFAR-10 Batch 5:   loss: 0.654554 accuracy: 0.7146\n",
      "Epoch 194, CIFAR-10 Batch 1:   loss: 0.726462 accuracy: 0.7194\n",
      "Epoch 194, CIFAR-10 Batch 2:   loss: 0.653591 accuracy: 0.7192\n",
      "Epoch 194, CIFAR-10 Batch 3:   loss: 0.598601 accuracy: 0.7054\n",
      "Epoch 194, CIFAR-10 Batch 4:   loss: 0.59352 accuracy: 0.7278\n",
      "Epoch 194, CIFAR-10 Batch 5:   loss: 0.635034 accuracy: 0.719\n",
      "Epoch 195, CIFAR-10 Batch 1:   loss: 0.700176 accuracy: 0.7218\n",
      "Epoch 195, CIFAR-10 Batch 2:   loss: 0.641786 accuracy: 0.7204\n",
      "Epoch 195, CIFAR-10 Batch 3:   loss: 0.59519 accuracy: 0.709\n",
      "Epoch 195, CIFAR-10 Batch 4:   loss: 0.589205 accuracy: 0.721\n",
      "Epoch 195, CIFAR-10 Batch 5:   loss: 0.671022 accuracy: 0.706\n",
      "Epoch 196, CIFAR-10 Batch 1:   loss: 0.699996 accuracy: 0.718\n",
      "Epoch 196, CIFAR-10 Batch 2:   loss: 0.655823 accuracy: 0.7214\n",
      "Epoch 196, CIFAR-10 Batch 3:   loss: 0.59994 accuracy: 0.7188\n",
      "Epoch 196, CIFAR-10 Batch 4:   loss: 0.605089 accuracy: 0.7218\n",
      "Epoch 196, CIFAR-10 Batch 5:   loss: 0.644799 accuracy: 0.7218\n",
      "Epoch 197, CIFAR-10 Batch 1:   loss: 0.677521 accuracy: 0.7226\n",
      "Epoch 197, CIFAR-10 Batch 2:   loss: 0.644166 accuracy: 0.7204\n",
      "Epoch 197, CIFAR-10 Batch 3:   loss: 0.576334 accuracy: 0.7252\n",
      "Epoch 197, CIFAR-10 Batch 4:   loss: 0.590224 accuracy: 0.7242\n",
      "Epoch 197, CIFAR-10 Batch 5:   loss: 0.622987 accuracy: 0.717\n",
      "Epoch 198, CIFAR-10 Batch 1:   loss: 0.71274 accuracy: 0.7244\n",
      "Epoch 198, CIFAR-10 Batch 2:   loss: 0.671852 accuracy: 0.7136\n",
      "Epoch 198, CIFAR-10 Batch 3:   loss: 0.596578 accuracy: 0.7072\n",
      "Epoch 198, CIFAR-10 Batch 4:   loss: 0.578598 accuracy: 0.724\n",
      "Epoch 198, CIFAR-10 Batch 5:   loss: 0.626314 accuracy: 0.7112\n",
      "Epoch 199, CIFAR-10 Batch 1:   loss: 0.729207 accuracy: 0.716\n",
      "Epoch 199, CIFAR-10 Batch 2:   loss: 0.629876 accuracy: 0.7194\n",
      "Epoch 199, CIFAR-10 Batch 3:   loss: 0.587942 accuracy: 0.7128\n",
      "Epoch 199, CIFAR-10 Batch 4:   loss: 0.579497 accuracy: 0.725\n",
      "Epoch 199, CIFAR-10 Batch 5:   loss: 0.663474 accuracy: 0.7044\n",
      "Epoch 200, CIFAR-10 Batch 1:   loss: 0.702851 accuracy: 0.717\n",
      "Epoch 200, CIFAR-10 Batch 2:   loss: 0.650691 accuracy: 0.7186\n",
      "Epoch 200, CIFAR-10 Batch 3:   loss: 0.608023 accuracy: 0.7128\n",
      "Epoch 200, CIFAR-10 Batch 4:   loss: 0.588716 accuracy: 0.732\n",
      "Epoch 200, CIFAR-10 Batch 5:   loss: 0.659427 accuracy: 0.7098\n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "save_model_path = './image_classification'\n",
    "\n",
    "print('Training...')\n",
    "with tf.Session() as sess:\n",
    "    # Initializing the variables\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "    \n",
    "    # Training cycle\n",
    "    for epoch in range(epochs):\n",
    "        # Loop over all batches\n",
    "        n_batches = 5\n",
    "        for batch_i in range(1, n_batches + 1):\n",
    "            for batch_features, batch_labels in helper.load_preprocess_training_batch(batch_i, batch_size):\n",
    "                train_neural_network(sess, optimizer, keep_probability, batch_features, batch_labels)\n",
    "            print('Epoch {:>2}, CIFAR-10 Batch {}:  '.format(epoch + 1, batch_i), end='')\n",
    "            print_stats(sess, batch_features, batch_labels, cost, accuracy)\n",
    "            \n",
    "    # Save Model\n",
    "    saver = tf.train.Saver()\n",
    "    save_path = saver.save(sess, save_model_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jm94DutayGVU"
   },
   "source": [
    "# 检查点\n",
    "\n",
    "模型已保存到本地。\n",
    "\n",
    "## 测试模型\n",
    "\n",
    "利用测试数据集测试你的模型。这将是最终的准确率。你的准确率应该高于 50%。如果没达到，请继续调整模型结构和参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {
    "colab": {
     "autoexec": {
      "startup": false,
      "wait_interval": 0
     },
     "base_uri": "https://localhost:8080/",
     "height": 392,
     "output_extras": [
      {
       "item_id": 2
      },
      {
       "item_id": 3
      }
     ]
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 3109,
     "status": "ok",
     "timestamp": 1518437767485,
     "user": {
      "displayName": "Jachin Shen",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s128",
      "userId": "106689561339379028936"
     },
     "user_tz": -480
    },
    "id": "S6-7BWW4yGVU",
    "outputId": "f3dcbb1e-efe3-4e70-d335-b41b3a73b5d2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from ./image_classification\n",
      "Testing Accuracy: 0.6978056073188782\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsUAAAJ/CAYAAACQmq4LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAWJQAAFiUBSVIk8AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3XecpFWV//FPde6ePDADOAMM8SBi\nAjGhPwEVwV1EXTGiiLqyQV1dE66rguKaw5owoYiigqggyIoRUHRlzQE4pBnJwzDDMKlThd8f51Y9\nT9dUp5nu6e6q7/v16ld1PfcJt6qru0+dOvfeQqVSQURERESklbXNdAdERERERGaagmIRERERaXkK\nikVERESk5SkoFhEREZGWp6BYRERERFqegmIRERERaXkKikVERESk5SkoFhEREZGWp6BYRERERFqe\ngmIRERERaXkKikVERESk5SkoFhEREZGWp6BYRERERFpex0x3QMTMngz8O/AEYCkwDNwDPNnd75nJ\nvonsKDO7CnhKunu1ux/dYJ9K7u5Z7n7m9Pds4sxsDbBvuvsVd3/5jHVGRGSaKSiWGWVmLwW+AhRy\nmzuBA4DuGemUiIiItBwFxU3CzOYBTweOAx5GBJULgT5gCNhKZF9vAn4JXObuN89Mb4OZ9QGfYGRA\nfEP6WkL0W2aRuuznRGwBHgBuA64FvuPuv52GromIiOyUQqVSGX8vmbXMrBN4M/AmIpCcqApwKfAG\nd18zDV0bl5k9A/hBbtP5wGnuXh5l/+cAjwTWuPt5099DqbcDQXEjVwOvdvebdr5Hs9dEyidmipk9\nGjgJYLaVbIiIzBRliucwM1tKBLZPqmsaAK4H1hIZ4h5gT+ChwLy0TwF4NvD/zOyZ7v7rXdLpkfap\nu/+B0QLi5J3Ao4ig6rzp6pRM2DDwnTHa24FFgDHyZ/0U4HdmdpK7/2Qa+yejexnw+vT9mTPYDxGR\nWUNB8RxlZgXga4wMiH8DnAX80N23Kz0wsy6ixOIs4Ii0eSlwiZk93N3vn95eb2ePuvurR9vRzOYD\nD5/e7sgkbXP3F05kRzMz4O3AS9OmecC3zexwd79tujooo3riTHdARGS20ZRsc9fxwAm5+5cCT3D3\nyxsFxADuPuTu3ydmechn+PYkApZdbcSbMnfvH2PfxxOZR5mDPLwMOCO3eRHwnhnqUssys17g0TPd\nDxGR2UZB8dz14tz3ZeBf3b04kQPdfRg4Dchnhl+R/lnOVkfNdAdk57n7B4D8QLvnm9nCmepPi3os\nMcOLiIjkqHxi7jo49/1qd79rMge7+yYz+2/gMcAf01dh7KPAzA4HngccQ9SJ7kbMErEeuBn4CfAN\nd799lOPPBN41Slv9qM/TgC832PUpdfuOmD+1ru1Ed788lZs8G/hn4HBgMVFvfTPwPeAT7r6xrj9H\nAv8G/D+i1GMIuDM9xo+4+6jlHnXnWUA8Z8cDj0jnWgAMEs/bH4EfAee7+4OjnGN34K/A8rTpRuCR\no30qkDvuKODnZD/bi9z9BRPp9zT6Gln5TgdRY3xZfgczOw84Nd291t2fZGYdxHzWLyNmVykBR7r7\nDY0uYmY9wIuAZxCv8+VEMLgOuAu4kng+rp9ox1MfTgGeT5TzLAM2pfP9EPiyu984ifNNap7iNGPL\ni4BnEbXaewFdxBvc3xHP4wWNPnVp8PvVaPvf3H1Vrm0Nk5inOD0/1df644jnZyHwYOrjdcTzfvEE\nXrtXkQ1UvMDdT0nbdwdOB54LHEjMsLMJuIX4GXzK3deOde50nk7gROLvwiOBlcTvZQnYmM53LfBN\nd//DeOcTkblPQfHclZ9pomtHTuDuZ090XzNbBnyW+EdUr5v4Z7KKqFl+t5l9GnhrykrPqBRIXAo8\nra5pIRGcHQG80syOdvfVZtYGfBr4p7r9u4BD0tdpZvas8QaKmdnJwKfIgtm8DqK2dh/in/NZZvZq\nd7+4fkd3v9/M/gWoth0CvAUY9WeYApRzyALi+4B/Hau/u8gf6+7v23Cv7X2DCLjyGs5lnZ73TxCl\nQfX2Tl+PB95pZp8jXqubxrq4me1HPP+H1zUtS1+PAt5kZu9297PGeSyTZmbHA18EVjRoXpm+nkW8\njl7u7j+a6j6M07/jiN+bAxs0756+DiHe1LzXzF7n7pc12HesazyLeB301TUtJTLgjwVea2bPdver\nxjjP44j50a1BcyfxutmTGLPxVjO7iJgxpeGbVhFpDiqfmLvypQ8rzewR03UhM9ubyJjkA+Iy8Hsi\n2Pw+4Lm2LuANwPfS4L68vwAXpq+/1rVdWPe1Ovf9utx+6+r2G2/mjG8RAfEw8AuinvpX6X7VPsBX\n0/fnkAXEfwC+C/yYyEZV9QFfN7PFo13UzE5J/csHxOuI2TO+C1wB5DPqS4ALzazRGw/c/dvAN3Ob\n3m5m+492fSLLnR+c+E8zMJiykS119xeMd4CZPY/tA+LR9n0bcBEjA+I7ief7UiKjWkrbC8TP+so0\n1/do51xKZDjzAXER+F+y18d64m/qmWb2lon0daLSa+n7jAyIbwQuJz7puCW3/SHAFWb27LrTTOT3\nblJBaoP+5QPiB4GfEb9vPyHmq65aBXzXzE6bxDVOTOeqZoZ/ku7/HMhnxhcBF5tZ/UDe6nkekY7N\nB8T3pm3fJn7O9Z8ePB+4PGWXRaRJKVM8d11LDJiD+Md+kZkdP9VzDudmuTgot/lrwJvd/d66fR8O\nfJ7IwEF8hPp2cuUSKQt6cdr/TGKhkWpbo5kMrk77XkX2Uer1E531AHgJ8EwiIHpVftnoFFD+kPgo\nHuAoMzsDeDURvJ/i7n/M7b+ACJxPSpuWE7Xdn6m/qJktSdurWdoHgVcA36uv/TazE9J5dyOCqnPM\n7Ep339rg8byGKF3Zg5hq79OMHHBZPedKRk61dYG7f7fB+WZC/XzaGyZwzBuJubU/TDyvdxHZwRHZ\n3RQ4vTe36Tbgn939h3X77QV8kCiFgHjNfp54vTTyHkb+DvwIeLm73507ZxtR2vAZ4N3A3UwBMzsM\nOJcsifFX4NT6RVBSpvbrxOuoAzjPzA5z9zsh+/2a4O/dZPr3MCKDXf1/spWY7u28/Gs9PT+nEK/Z\n+cTA2XPM7FcTKDlZSvx8ysBbgU/myy/S79uXiFIIiOfgdOLnUO/jZFNT3g+8zN3/p8HjWkW8Rk5O\nm54E/CMNft9FpDkoUzx3fZaRmU4D/mJm7zezg0Y5ZkecQtTUVn3F3V9aHxADuPufifKJfNb4jBSA\nzJQXElnhZ+cDYoA0Fdjr6vZ/H5HNfWo+IE77bwZexcis1PGjXPcljMyAvtHdv9NoMGT6h/zq3Kbl\nNC5Twd3XE//sa9c3s+c32PXjROABsZLha0fp50yon/lgvIDoQCJofYu7v8Xd17j7sLuvzdfOpize\np8jeiNwOHFUfEAO4+z3u/lIimKt6sZk9vn5fM9uTkT+fG4jX04ig193L7n4B8aapC9hvnMc1UR8l\nK5G6D3h6o1UB0+N8GfHmASJj+tYp6sNYPkFWxlIBnuXuX6x/rafn53zgH3Kbu4EPTeAaJxCZ/1Pc\n/SP19cju/gDx2NfnNp9Yf5JUj5xffOb1jQLidM417v58olyj6lUT6KuIzFEKiucod7+VqCnNm0f8\nE7zJzG40s8+Z2ctSLeSOytfVbiYGOo3Vry3Af+Q2dTF69m1XeeMYtc0/IhY7yXu/u9/XaOdUfvDL\n3KaHNdqPyECdT2Si/0h8LDuWSxiZ9XzyaDu6+6XABblNH8vP4JBqT/OBx6tT0DDj0icPL81t2ky8\naRnLHsCtRHA4lhcycpGQ1zR681bnjYx83uvryCGyj/lP1d7r7ttGO2GqZb1wnOtOSJrf+em5TR+t\nf3NXd+0rgP/LbXpxytBOi/QG/Njcpq+7+0/HOiYF7/kyjRNGK3Woc5m7XzTGeTcTnwhVHdbgsa9i\n5P+930zguu8mavf/kVg9VESalMon5jB3/7iZDRLBQk9ds6WvVwOY2Z3ANUQQeMVoQd+IE0SGNz/J\n/6XuPpGPui8nAo1qoPYs4mPvmbDa3UcNutx92MxWE6v9VY0X0NwEPDV9v2yU836TkfW/Y3L3spnd\nRMySADGrwFheSwQjexE1pGcDr0szLnwqt9957n75RPuxC7ydkW8kPjPeLATJBT72aocwMrt+50QG\ncaVZWC4iywCeaGbt7l7K7XZc7vsiE6u7PZ8I0nfWSXX3x3tzBfFaeBTxxux+4o1p/Ru/qfIPdfcb\nzRbTyNfJMrntwN8R5Q9j+fwEzpuvBe4hsuX5N4SlkbtzKCM/2dpOKu14xwSuLSJznDLFc5y7n0N8\nvPxJRn6sX28lUf/6ZeAeM7vKzE4aJ4t0RN39n02wT0OMzMBM2yDACbhuAvvkp2K7ZwLT2+X3H3Vw\n1g7I1xDPH3Uvah8X5z/S/5c0Xd5/kNVI30m2lO+MMrNVZvZFRi7WsRr4wARP8YsJ7JNf3fGaifaN\nka/rpWTPX9Whue+vH2+WiuTnk7j+WPKPaYO73zLqnom7X+bu73H3c9z9W+4+XQExZG/iIN4wTPRx\nX1t3/5Hj7F+Z4LnrPxGp/z26hZFvED5hZvV/50SkRSlT3ARSEPc6M/sPYlDZScDRRAaxkTairu4p\nwPVmdpq7Nwoe65dVrh+1PpYbyD5WXWRmKyYQbE6HOyewTz5TOZHBUfn9J/TG0swOJQZGHkRMTdXX\n4Nh88DXunNEe8y9/hZjPt50YrJcP6F41zVNI9ZnZWNnwAvFpwf6MnFcb4nk+cRJlHTeN1Zg+ft89\nt2nceWpz6muaH1a9XnrTmH9O10zkhO6+xczuIKZ+2xn55+1vO3mu6ZD/G3HLBLP+uPsdZraFLGg9\ndKz9gY0TfC0P1t0fsQqmu282s3PJpiZcCfyfmV1JDAD+sbvPxudZRHYBBcVNJNXzXpS+MLMDiEzT\nUUSN6iENDjsUuMrM/r5BLeDSuvvjllzkrK+7v5iYMWBXazSDw1TuPyYzO5UoGZjKwY95ryemm1vB\nyMDii+5+5TRds6oTmOxCIBViCrN/mcgCCznrxmmvf62+wczeMKmeZfJTuc1n5N/JyUxpt56dD4rz\nfan/nZoN8s/7ZP4+QMw6Ug2KR53aMNk8yXOP5U1EaVl13vICMWD2eAAzu5WYYu8HwJU+9vLzItJE\nFBQ3sTQY71ZikvrqKPpnEvWTT8jt2gt8w8wOqvtouH753fr5ZcdSv+9MLeXbcBWvKdy/obRwxleZ\nmrrSUbn7RjN7M1GjWbWVGEA2GwwQwdxtRJnCRe4+mU8cqsYLTMYLqiYjP2tI/cfvkwmQpuINVr48\nZzrLIHZU/vd6Mn8f6vcf7+/DlPxeArj7QJoG8XXA2xj5CQPEJwMHELO8bDGzC4Gzp3q6SxGZfRQU\nt5A0Ev9LwJfSil/nka0MtZyYR/fjuUPq/xGN+5F+Tn1pwHiDpJrN2YwMiAeJ5/u7RF3jBuIj4dpz\nXDcX82S8qO7+PCKD+4UdONdkPOjuUxmMjir/PI2ifgDVHxhnANUYbs19X/+an0xwNhV/X/O/N7Nx\nDEj++ZjM3wcY+Xh26d+HNF3cR9PKm3+fvo5n+xUQ5wOvBF5qZqe7+3m7sp8ismspKG5R7v4tM+sm\nW8UN4BmMDIrra/jmM/GPSOszbC2zPGqaCzU/dd1m4Fh3n8j0T5O91ktpMB8r8JG0AMjtDdqaUf3r\n62J3f2/DPSenPttbP8vLWKZiEOZmYiGKqTrfVHuQbMXGMQeHNpDff0b+Prj7IDGjx7fTdIGPIoLj\nE4iZd6o1yV3Al82s4u5fmYm+isj0m42ZB9l1vs7If0b71LXX1zA2nH5sFPX7TmQqt2bxHKLetupd\nEwyI6+tix5TKYf47t+m9RIYUogTg3PSPvhXU1xwvb7jX5G1mZDa0fjW+sUzFojX5uuvRBs7OpPzf\niAn/fUgDGHfLbZrxvw/uXnH337v7+9z9/xF/Dz/JyJ//BxssXS8iTUJB8RxlZsvN7Jlm9qTx924s\nzfuaX9yg/qPhP9XdP2wSp8/vu3Yi8yI3kfopvb4z3gFmNp/GAyHH8lmyIO1GYrqzV5N9FP00Gi9G\n0XTS/Nn5mUYm81od67wlYmW8qvo3jg2Z2TJGBn07Kl9/vW/6dGc2yf+NOHAS/dufGMtQ9eep69LU\ncPe73f11wH/mNi8HHjdDXRKRaaageI4xsz4zu43IIH2f8Vf5GutcbYysobujbpfrGBkoH8sEpADv\n8NymKS8bmOXqg6HxVlWDmEO6c9y9EjN7CdnCDhVi1bpBd/8/Ri7e8cGdXNFwLvl17vuj8qv87aT8\nghCPSAukjOdp4+8yIfmpEruI5a7HZGZPT/OQV7/q36RNpfxz3sHIJeHHUr/f/zXca3b4XN39lTPS\nCxGZdgqK55i0vGx+INCRZva8HTzdicSKT1U/rLvWBuAnuU3PTvWy43k+I7NAoy7NuoMmHDzOkPrp\no+oH74xgZrsB76rbPOpjTHPyfiK36Qvunl/Y4D/JsqbziYGVrVBG8Y3c993EsrzjMrNTU/D4FjNr\ntGz3VXXnPX4Cpz11IteegG8x8o3paRM45mSyecgfw/Zvdkcws535ffo2IwfJvWq0Heu8PPf9ZmL6\ns2llZu1m9jYzu9jMVpvZY8Y/CoD6JeInsniLiMxBCornprPq7n/JzI5ruOcozOwQ4uP3qgeBCxrs\nms869jFOZjp9bHx2btP9xD/2nZVfFGA21lbmXV93/4TRdjSzBcSMFHsCf8k1jRVIn0NWf3wv8NZ8\no7tvJpaBrjoaeM2YPW4O32NkAPhOMxuzjMLMDgTeTwSQH2Dka7eqvvzlHWPVlZrZs4lBqzstLSTx\n/dyml5jZE0bb38wOAk7JbfpmgwU16u/v8O9TGsj5vdymk83sqaPtn/p4CjFvetX5aY71aZVKYZ5P\nLE29CvivCdYH55cPL5PV7YtIk1FQPAe5+y8YuTzuAuB/zOwCM3vSWFlBM3uomb2PKGnIB17/1qju\n190vZeQ/5Zea2ZcbZYzN7CjgakYOMPrXKZr8Pl+CsCpNKVe9bl+D/WfS/xBL3ladbWYjplozs44U\nPP2BCBA+CVyR22X/RpksM3sRMZCv6rXuvrF+P3e/BLgkt+n90/wx+oxz92FGLn29EPiZmb3QzEas\nbGZmXWb2MmL56OrvwSCxTHb9eW8hVjurOhy4qP53IP1MTycGsA4zdcHTG8lWausALjezZ9X/npvZ\ns4hFJ6qf0mwD3t3gfPXlPG+qLveeHsNkM8f/TjbncAG4xMxeVr+EfHrOX09MC1l1N/COSV5vZ3w4\n9/3TgStHe+OUnotTGfmpzOUztDKniOwCmpJt7npbun0L8Y+ojahLfTEx4fwNxIj8rcRHvrsRS8bW\njxAfIgLXsaYZegVRRlH95/FyImP1W+KfWi+xmtq+dce9392nqnTiGuClufsXmdlaIvjoY2oGNU0J\nd7/LzD5HtpTsbkRw9lei9GUR8VxWg6pfEMHYM+tO9VMz+zlwv7ufambLieC56jJ3v5jRvRZ4KvGm\nqY+YUuroNMCyKbn7D8zsrWRvGncnyio+YWZ/Ij6qfwixwmB+JokS8Ep3v2GUU7+JqIOtzmpxEvBM\nM7uOqO9fBBxBtojIWcTAyUdNwWO6Kb0Z+hYxRdhS4FLgb2Z2fdp2CCMHAZaBl40yJd81dfdfA5xq\nZuuJN7SPZxIBvbuvTlMDfpP4WzOfWDDoo+lvxBbi53A4I6dhewB43iSW+p4KXweeRWSMIT5F+bOZ\n3U4MVt1E/D3bnfibll/I5U6y32kRaULKFM9RafqgM4BjgJ/XNc8HjiSCrJOJfwJHMTIgLhKZxMPc\n/YvjXOs+Ipt5EVl9Yyfxz/O5RHlAPiBeRwQYb2PqXMD2izHsQQx6ad9+9xn3JkZm2AtEIHwS8Y+4\nGhBfAhyXasUvB1bnjllA/AyrWebPkAX/m4F/GasD7n4nscR01ZOBf5vk45hz3P2DxOIl+YzeMuIN\nwrOBxzIyIL4NOMHdG5UPVc/5N6IkIl/P30n8Xj03nXsx8fvxUSIonrI3H+7+3XT9/OtjX+J37zhG\nBsR3Ac9w92+Pcq5bGDk/OcRrbRUR1O5I/y5J/bgpt3m3tO25xBuKfEB8LfBEd//VjlxvR6VFYE4B\nPsTIT3P2Ifr6PODviBkm8gHxlcBR6XdKRJqUguI5zt2vTnNqHkJ8jHkhkeVZR3x8WiYyNXcDvyU+\nuvxnYIW7P8fdb57gdTa6+wuIbNgHiVHx1UztJiJY+DaxNOr+7v6l0c61I1IJxjFEBuq+9Li2ElNW\njRrMzBR3HyDLSF1OfGQ9TCwTfAvwNeDp6WfQnzvm6Wn/TcSyvrcSS3C/gKiFrHr7BP9Bf5qRI/vf\na2YH78xjmwvSJxQHEgPTLgRuBjYSgdBG4AYiMHwecIi7/2gC5/wD8cbmNcQnJ3cTn7TcT0xN9nHg\ncHd/Ywq+pnTuXXf/CfAw4hOT76bHtJkorbgLuIwYXHiAu/94nNO9kpjC7zbiORkE1hBlImtHP2zM\n/l2T+vdi4vV9E9lzvg74I/FJx9Pc/UnufuOOXGdnufuwu78F2I+ox7+C+D3bTHxiMED8vl5DBM9H\nuvvxLbQQjkjLKlQqU7akvIiIiIjInKRMsYiIiIi0PAXFIiIiItLyFBSLiIiISMtTUCwiIiIiLU9B\nsYiIiIi0PAXFIiIiItLyFBSLiIiISMtTUCwiIiIiLU9BsYiIiIi0PAXFIiIiItLyFBSLiIiISMtT\nUCwiIiIiLU9BsYiIiIi0PAXFIiIiItLyFBSLiIiISMtTUCwiIiIiLU9BsYiIiIi0PAXFIiIiItLy\nFBSLiIiISMtTUCwiIiIiLU9BsYiIiIi0PAXFIiIiItLyFBSLiIiISMtTUCwiIiIiLU9BsYiIiIi0\nPAXFIiIiItLyFBSLiIiISMtTUCwiIiIiLU9B8SjM7OVmVjGzqyZxTCV9rZrCfpyXznnmVJ1TRERE\nREbqmOkONJn/TrebZrQXIiIiIjIpCoqnkLu/fqb7ICIiIiKTp/IJEREREWl5yhRPgJl1A+8Ang/s\nA2wFrgHe6e5/zu1XSd/u5+5r0rbzgFOBNwPXAx8B9gee6u6/SPvsAZwN/B2wG3AvcBnwzml+aCIi\nIiKCMsUTdSnwKuB/gfOAdcCzgWvN7LAJnmMv4JvALcC5wAYAM1sCXJvOXwAuAH4EHJ+2903VgxAR\nERGRxpQpHt8Tgd8CB7r7FgAz6wC+B5wAfIDI8I7n5URm+eN1298GHADcADzB3R/MXeOrwMlT8BhE\nREREZAzKFI+vE/j3akAM4O5F4Kx09zgzWzSB83QB5zTY/uJ0e3Y1IM5d47VApcExIiIiIjKFFBSP\n7wF3/1WD7b8Bhohs+0RKKH7l7oP5DWa2J7Ai3b2m/gB3vx/43eS6KyIiIiKTpaB4fKsbbXT3ErA2\n3d1rAue5r8G2Fbnv7x7luL9N4NwiIiIishMUFI9v6xhtA+m2ZwLn2dJgW3UQ3ZC7l0c5bnCU7SIi\nIiIyRRQUj2+sgLfatm0Hz10NqjvNrDDKPpp9QkRERGSaKSge3z6NNqbZIaplE3fu4LnvSbcFYI9R\n9tl/B88tIiIiIhOkoHh8e5jZIxtsfxwxyG4Q+OuOnNjd7wTuT3efVN9uZiuAR+zIuUVERERk4hQU\nj28I+LiZ9VY3mFkn2ZRsl7r7WHXH47k43b7NzGqlEukanwSKO3FuEREREZkALd4xvqvS7S1m9iMi\nM3wscCCwnlh8Y2e8h1gd73Dg+nSNtnSNYeB8YrU7EREREZkmyhSPrwicBHyFKHE4FVgCfItYge62\nnTm5u98NPAH4GtCbzn8ccCWxmt6GnTm/iIiIiIyvUKlowTQRERERaW3KFIuIiIhIy1NQLCIiIiIt\nT0GxiIiIiLQ8BcUiIiIi0vIUFIuIiIhIy1NQLCIiIiItT0GxiIiIiLQ8BcUiIiIi0vIUFIuIiIhI\ny1NQLCIiIiItr2OmOzCVjjh6ZQWg0JbF+n1dvQB0tPcBUMm9D+js7AZgXt88ABYsmF9rW7RwQRzf\nE/u0585ZKMTT1j84CMDWbQO1tmKpBEBbe3s6rlBra2uvflfOOl2I9vb2OOf81CeABd1dACxfGP1b\nOq+z1rakL+3fGyed3531r6MQS3cPDUdfhnOX2zo8DMBRz3tP1jER2VmVdes2z3QfRMa0bFn8X9Nr\nVeaC9HrdpbFKUwXF7SmgbMsHsCkSLVciUCyloBWgPe1W3VTOBY9pd0qltLGS/VwKlTigNBRtpeHs\nnKXaSSrpPJVaW0chLthWqEXHFAqd6ZzxoxgezK4zlM4xkKLawWJ2ri2DEdwOF4fiOqUsYO7pjHMN\nDEW/hopZ//qLw4jMRmZ2KvB+YHfg7e7+wRnukoiItJCmCopFZG4ys6XA54ANwD8Bv53ZHomISKtR\nUCwis8HBQDdwgbufO9OdmawT33jpTHdBZNK+dMaxM90FkVmlqYLicrVeuJCVT9QqDkrF2KeUlSd0\ntqcSh3JsKxTy4w5jW636IVcGUSmX0v5R1tDdmT2NnalGuExcr5KrH66WdXR1ddW2dXT05LtHJVfd\nUEp9HUoPotyelUgMVdLjSaUb3fnjUv+2bIua53Ih63uhPSvdEJlFetLt1hnthYiItKymCopFZO4x\ns6uAp6S77zKzdwFfAU4FzgWuBt4L3Ovuj03H9ABvAV4I7AcUgeuBz9dnms1sD+BDwDOBXuA64E3A\ns4H/BI5x96um7xGKiMhc0FRBcbFczfzmBr6lQXFtKetayD3kckq8VirbZ4rLpThXe0/s39mRZVjb\niHO2d0TGt9iVZYPLKVM8VIr9R2SKU5K6tzebYaKjMxJkgwOR+S2Uc1ndtrh2Jc12Mbz9pBVUZ9Ub\nHM6OGxqMwXdb+yN93NaRmwEjSzaLzBbvAo4GzgS+BVxEBLinAiuBs4CzgXsAzKwNuAx4GvBN4GNE\n6cXJwBfNbD93/8+0bztwJfBI4Dzg58DDgB8Av9wFj01EROaIpgqKRWTucferzaz6zu16d7/YzFal\n+8cBT3b3a3OHnEwExJ9399OrG83ss8BvgDPM7Bx3vws4iQiIz3f303L7/g742rQ9KJE5oDpFm4iE\npgqKOzojDTowPFjbVp2XuLePG9V1AAAgAElEQVQjHmpXe2+2f1eaDq0zZYjbs2xre2f8j+7pjqxu\nT1eWKe5oS1OypanOBqsFwcBwyjC3leJ6hbbtp9jryNUgt7XF/h1pfrhKZ9aHtpSdLqTb/tzUapWU\nUW5vj/P3D2Vp5GLKFA8OR7/a8hnmXBZdZA64py4gBnhOuv1sfqO7F83sq8CHgeOJ0otjUvPX687x\ndeB9wN5T210REZmrmiooFpGms6bBtkPS7V8btHm6PTjdrkq3t4zYyb1iZr9GQbG0MC3iIbPZTHyS\noWWeRWQ2a/Rfez4w7O5DDdr60+28dNuXbrc12PfBneybiIg0kabKFHemcohyvgwirRjX2x4D2qrL\nPgP09sWyzl198X+zb0FPri3KJrrTMsqd7dn7h/Y0eK49TdtWyLWRShwK6f1GITdAr7YoXq6iolyJ\nc1UX4WvryfbvTKUR1VKOUiUr06iWT1TS+QdzK/UNpZXs+ociZijkRui1d2mkncx5W4BOM+tqEBhX\ng+FqMF2tpephewuno3MiIjI3NVVQLCIt4Xpi8NzD2X7lu0PT7Q3p9q50uy+wurpTGtj3uKnq0GUf\nOUkfRcusV/04Wq9VkcaaKijuTYPiutqzh9XTFVngnvaYPq23M0sYzZ8XmeK+Bem2J5cp7opzdNTW\n2cgNUKuMzBDnp1FrS4netupiH7kkcnXGt3JumjZSprgzDfZrb88P6EvXSYPvcjOrUUjnKLdVFxnJ\nzjmcMsqDw2lKtmz+NjpVMCNz37eAFxHLQf9jdaOZdRPTuA0A30+bfwm8Cng+cFXuHC8B9tkFfRUR\nkTmiqYJiEWkJlxBB76vSIh4/AxYQgfIhwL+5+/q074XEPMf/ZGYV4P+Aw4AXAJcSU7aJiIhooJ2I\nzC3uXgGeC7wTOBI4B3gPUT/8HHf/RG7fbcScxlcSWeSPAgY8HVibdtM8hSIi0lyZ4vnzYoxNW3dW\nLtDTHSURXWme4t72bKDZovkx6K53fhpU15mtNFcdO1egukpebtW6VD5RLUso5EoeqtMS18be5UoX\nimmO4HJuruDq+avnaM/1r5BqMbp7UwlI7qc1PLgFgIHain25gXZpruThSirJIOsfZb0PktknLbNc\nyN1fw4ghqdvtP0QEwu+ZwLlvAk6o325mu6dv759cb0VEpBk1VVAsIpJnZiuA/wZudve35bbvQayW\ntw64aYa6JyIis0hTBcXVTHFnX5YZbU8p27602t38ztrIOZYvjRmZ2jqq06dlWdRSNdtaTAPmchnf\n6jRqheqUbLlV66rj6yhUM8BZ/6rZ5kru09qO9uqx1XNmmeLDDnoUAAet2i91YbjWtvqOWItg9Zo0\nyL6UDfYbStngoXL7dv1rL+YG+Yk0v7uBPYF/SIHwz4AlwGuJ+Y7f4u4qnxAREdUUi0jzSvXHzwQ+\nADwZ+DzwXuA+4AXufs4Mdk9ERGaRpsoU93RFFri64AZAX5parZAyqfN6s0xxX2fsVy2zreTWtSik\n+c9K5bgdLubqgKs1walmt5IrfSxXF9Wo1grn3ne0VbPChexpb++o1idHX1bssbLWtmxznMMvuRKA\n3Q98SK1tt732AmBd7wYAtpUeqLWVKlvSbcpM5/Jg5cKoZZoiTcndNwFnpC8REZGGlCkWERERkZan\noFhEREREWl5TlU90tkWM39OZlU8ctNduABQHooagf3io1lYtZxhO5Qz5qdIqhfi+ulBcsZQdV0iD\n1UqlOL7cll2vWKmWT6Q+5frXkUokymSD4gppQF5HWnlv1V7719p6fx6D4jtv/RsA81YurbVt2xqD\n7rZuSiUShexKlTT9XKEjzt2WW3GvXeUTIiIiIttRplhEREREWl5TZYrbUsa2kh8UN1AE4KF77gPA\n5uFttbYHB/rjdnAQgPZKbrqyQhw3NBTvGwb7s0xxZTDtlzKw5UL23qKYErFtaTGOfGK2Oj1ceyHX\nv1Kca37vMgA6KwtrbX3zVwCwcv894zrdi2pti5fE+Tu64vEMFAez5yEN3utMGeO23EBAJYpFRERE\ntqdMsYiIiIi0vKbKFFdS1rUylGV8d1sUNcUbNsU0ZX25hT16hyNjuzDVFBfas+OGUtu6lCHu35Jl\nisuD0dZefU/RkVvKubrMc6rnbc8vAZ1qnou5GuTO9CPYa97iuM6aNbW2rQMDANyxNva/s39trW1F\nZ9QUd/RFtrsjSxQznNb4KBS2f89TKVS22yYiIiLS6pQpFhEREZGWp6BYRERERFpeU5VPDPRHDcG8\nzu7atrZylB78fvUtABy0z/Jam+0RA9n6+mNQ3YbN99XaysNRZrB5a5QwVPqzsoPSUHyfFsujkBu9\nViJKMMrlOGd1sBtAR0f0pZh72veY1wPAXsVYkW7w3gdrbTeuj7ab7ozp2tpXba61dW+LAXbDqRyi\nWMlKMsq58YIAFbINpVIJERERERlJmWIRmTXMbI2ZrZnpfoiISOtpqkzx4GBkdfu6sqnL1q3dELfr\nI8v6hMc8uta2MWWBf/NXB2CvJT21tlIatDc4EKPW2oq5ac3K8X2lHO8pysUsi1wspKxsJS0MUizW\n2tqGI5vb05M97Q9fNg+A3TavA+C2rZtqbV/7XZx/7d2R0T7hwGyBji0pu11M061VivlBgqkLKSlc\nqWR9KJY00E5ERESkXlMFxSIiM+HEN146011oGl8649iZ7oKItCiVT4iIiIhIy2uqTHElDYBb0pHF\n+msfiPKJzrZ4qJsfGKi1/elPfwFgY5qTeO/lS2ptg0Mx/29lIJUbVIZrbW3paWtLFQul3Mi2ctpY\nSqvXDQ3lyicqse3Rq/aqbdu3IwYF3rV+PQBXr8lW3LvpgYcCsCVt2jC8odbWtS3mTe4qRMlHbjpk\nhirR92IqqchVT1A3Bk9kRpjZU4H/Ah4BbAN+BLyhwX7tafvLgIOBIeDPwGfc/YIG+78C+CfgMOLl\nfhPwZeDT7l5O+6wCVgPnAlcD7wXudffHTumDFBGROaWpgmIRmf3M7DHAFcAmIjC+C3g88EOgiwh8\nMbMCcCHwHOBrwEeB+cCLgK+Z2X7ufnbuvB8B/h24FPgc0AmcCHwCeCTwqrqurATOAs4G7pmGhyo7\nYNmyBTPdhaan51iksaYKihd0RVZ3nz0W17ZtTSvLLSjHALbC/PW1ti2DkW0tpUzqvPl9tbZt2yKn\n2lGMNG1XR25luvYY3NaepkErl7NpzjpLMS1cWzkGxQ2UemttBy+M8x/ZlQ3oG7rvfgCuWh0Z7B/d\n/vBaW6n7YAC6u34V1yObkm1oS2SDe+fH+fsHswz4cMpqD1Uz2KVskKCG2cks8B9E8PsCd/9p2vYl\nM3sH8G7gb2nbicA/AG9x9w9VDzazc4BrgXea2efd/T4zeyQREH/G3f81d63PmtnFwCvN7NPu/vtc\n23HAk9392ul4kCIiMrc0VVAsInPCU4G1uYC46rNEUFz1gnT7LTNbXLfvd4DHAUcB3wWen7Zf2GDf\ni4ng+mggHxTfo4B49lm3bvP4O8kOqWaI9RzLXDATn2g0VVD8+CetBGDvA/eobbt57d0AHLDvfABW\nLJ9fa3tg/cMAWHPLzQAcaNlxmzdGxrc3ZYjn9WQZ30qabq2cFsKoTt8GMFiMLHCxFMcdtHy3Wttx\nh0Qtce/gltq2a+6P/vxiU2SI1/fsn/Vh620APHTfLgA68j+tNB3cfRsi0zxQGcr611adpi32qS5g\nAtDe1lQ/cpljzGw3YCFRFzyCu68zs/W5TYem29VjnHKfun2vnsC+VWvG2FdERFqMIiQR2ZWqNUrb\nRmnvz32/gKj4eRqjjxFdndsXot743lH2ra8bVrpMRERqFBSLyK5UDXp7RmmfD1TXOt8MFIC/uPt9\no+xPbl+A29z9up3rooiItKKmCopXHhAlDu175VZ+2xij6PbZPcoYHtiSJaIG0lRshxwUbQvnZcmo\ngS0xMG/xgjhXX1dXra1ciadtaDiN0BvOrrc1DbDbf3Hsc8ojFtbaFqfk2HVD2barCrHC3r2Ld4/r\nrss+VV64LEoi9jlwKQAdfVli694H4/vBoThnuT2bd606rq69En3p7ZhXa+vu0tTUMqPWA1uB/esb\nzGwFsJgsKP4r8CiyuuH8vouBLe5ezO377LTvdXX7zgeK7j7ANLnsIyepTlNEZI5ThCQiu4y7V4Br\ngBVm9oS65lfX3b8o3b7ezGp/q9JUbV8D7jSz6jvMb6XbfzazXkb6ILDOzA7Y6QcgIiJNq6kyxcWN\n8X/zNxuyAeYDHfH/ccM9MU3ZDy7PkkjL58XAvAMPjUzs5vseqLW1DcenuwsXxOIa3Z1Zppg00G5w\nKDLNg5uzKdkWdUfJ5BMPjf+/mzZl5/zftTHg7ddDB9W2/fq+yE5vLq4FYL+Dsizyonmx/3zik+OB\nbdm5HtgSybRKRyTKKrnBfkWqi3ZEpjmXyGZhm94HyYz7APAM4Ntm9imiBvgJxFzFq0lv1t39e2b2\nXWKe4h+b2VeJuYdfCBwDnO3um9K+fzSzjwOvB641s88Bw8DfAc8Fvubut+7CxygiInOMIiQR2aXc\n/Woi0L0HeCfwIWARcDxRXpH3fOAtwDLgM8DHiUF1r3L3d9Sd9w3EAh1F4GNp/4PT8adN08MREZEm\nUahUmmc5h29+6h8rAFffcWNtW8/CmLK0b1vE/3+86bZa23NPOg6AY46IjPGdN6+rtf32urRox8KU\nMe7JMsVdnZGJLabi3Vv/ltUSXn9z/E9f1BcZ6t//+YZa27bFkSFu3+NR2baFsbT0PgceCMDu7Q/W\n2tjwh9j/nj8B0FnYWGvqb48+DBVi6rhyR/ZzLJbT96V4zL2d2aIki+bFIP1zP/rNbEUPEdlZFdUU\ny2yneYplLkmv110aqyhTLCIiIiItT0GxiIiIiLS8phpo57fGOJoNG7KBbz33RBnEwiUx7dp+y5bU\n2kr9Me3a5v6Ypm3J3tnqc0vvjkFq3Slzv7S7u9bW3R2lFOXOmOFpuJhNuXrhpTGl2i13xUpzyw6x\nWttBR8SCWrsvzN6LtHXEdSrlWwDYY3Btra2yNT2egfioq9idrUxHJQ2mq8RjLZez8olSGghY3buQ\nq5ApFbPnRkRERESCMsUiIiIi0vKaKlPcviwytu0PZgt07LMkFsV4uK0CoFRZlLUdsAKAYlukUju7\nsgUwVu4fGdXKhhik1lfIpjzbWorzt/XEohhDndnTuHhlnPOJh0RWeOWqfWttlbbI+PZuzaZW29If\nA/PuXH0XAHsvyRbaWLY8porb1hFzqg2XsyxvsRTf12ZYa89lkQspU1yIbZVSdlxpOHuMIiIiIhKU\nKRYRERGRlqegWERERERaXlOVT/QuicFw++65e23bvEqUHixbGmUQne1Z+UB7Jb4vtMcqcjfdvCY7\n2UDMF1zeEKUV/T3ZyrHbemP/DffGtutX319rO+JxhwKwvC/O3ZVVXbBtw4a4Xm/2XuTuji2xrfQ3\nAOb3ZOUWDz04znX3lhhwt3kwK5FoT3Mkd6TbYjGbyq+cTl8hyiaGcuUT7RW9DxIRERGppwhJRERE\nRFpeU2WKly+KzG0pTcMGsHZdZHHnz4usa6E9W5nu6l//HoAVhxwCQDvza23r10YGtzMNsNu4JcvS\nbkqr4w0UI5t88MFZFrl/Uwyca+8fBiCXwKW/I7b1bx2ubVuyIgbTHf64RwCwrCvrwx77xPRxe6+P\nVfluXLOp1tbdlqaBG4iMdHEoS0kXC7GtVI7rFHI/5raKFrITERERqadMsYiIiIi0vKbKFC9eELW+\nbXtlmds9VsYUbLstjrbV69bV2m5bFzW+e+8fdccD27Kp3Dq7I0u7Yp+9APDbsmnUSuXIEO+3d9Qp\nt/Vn68jfvi7OsX7LIAD9mwdrbYPbYqGO/EIb5UK0H37EfgA8pGdhrW1JOv9em+Mx3Hp3dp1qxrtS\njL5v68+y48WUIa6kBT7a27Mschkt3iEiIiJST5liEREREWl5CopFZFYws/PMrGKWVtoZfb8z035H\n7+T1puQ8IiLSHJqqfOKb3/4pAI867LDatsceGYPoBlMJwXA5e8gbHtwIwE03OgDLHpJN5dadVpPb\n65Aoa9jSmw20e2BtmlqtP85ZGsrKISrV1fG604C2wc5aW1ca5DacK2HoTt3Zb2kMpiuUsv1vvP3P\nce3+KLtY3J0NEhwcjG3DnalfWeUHhXJcp7PQOeI+QE+73gfJnHcR8BfgrzPdERERaR5NFRSLSPNz\n9+uB62e6H3knvvHSme7CrPelM46d6S6IiIypqYLi4rZlAPzpxntq2+yIGJy2aHFMYdbH8lrb3z/l\nSQA8uPk+ABYvzbLBd90fU7nde/e9AGzYmJ3zgQdiarSO3rheR1dumrP0jPb0RJa2s5RlZiu9sa1S\nyDLL83ri2EXzY2DfXXdlAwE3borr9BVjwN2CjixTPFyMLHV1wF17e5Zh7i6nNlLGuD17XPM7exAR\nERGRkZoqKBaRptBlZu8HXgIsB+4AznH3j0DUAgPvAo5x96vStgrwE+BDwCeBhe6+Z2pbBnwE+Dug\nlyi7OGsXPh4REZkDmiooNjsKgL9c/7+1bX3E0s/bijH12Z/uWVNrWzgUhbiPPiiWVl62ZGWtbdX8\nyLxuvSumYtt8+121tge3xTn722LbHg/J+tAzL7Ky29IMbp25LPL8vnmxT3eWre1IU7J1p0U7hotr\na22Ll66Ib8qx/+rbs2x1VzUzXIqs855Ll9bayqWoN75vQ3Sit6u71tbbmWWbRWapjwFdwPuJv1Gn\nAh82s7K7f2yM4/qAzwCfANYCmFkbcAXwGOArwNXAirTfLdP1AEREZO5pqqBYRJrCPOBYdy8DmNlX\ngdXAGWb232Mc93jgpe5+QW7b3xMB8Vfd/eXVjWZ2IfDnqe64jG7ZsgUz3QVJ9LMQaUxTEYjIbPOF\nakAM4O4bgB8TpRSHjnFcCbikbttT0+038hvd/Wai3EJERARoskzxwkWxkt2jDjmitu3G390OwH2F\nuwHo2jZca+tNq8f95NoYyL7vPtlUacc+8RkAXHfNVQC0l7PBaou79wDg5ltiRqj+oYHcOZek/aNU\nojsb/waFKGtYND+b+m1+KpugFD+KeQuy1fiW7B0D+e66N5VU9BRrbR3p2/kdUZ7xiDR1HMC6+2Lg\n4AMbY8q5tkL23qdQyA0KFJmd/tJg223pdt8xjrvP3bfWbds/3d7cYP8bgGdOsm+yg9at2zz+TjKt\nqhli/SxkLpiJTzSUKRaR2abRf+zqTNy9DdrGOq4v3W5r0NbfYJuIiLSopsoUd1RiYNnCxYtr266+\n5pcAzFsa//8euizL0t68LqY/K5cXAXDNvWtqbZs3/ACAwQ2RYe7YbUmtbf3dMcBuzc2Rhd6yNRs4\nt/uyyBAvmBcZ2cW5dzrLl8c5OnJZ5z33jP6U0raluf51t0VWu5AGBC5ZksUDDw5H2wG7xWC8Q/bd\nu9ZW3BKxQXXRjuFS7ZNoSuXse5FZqm+MbY2C27FUA99GcxHOn+S5RESkiTVVUCwiTeGhbF9CcWC6\nvQ147CTO9bd0uz9ZCUbVYUyRyz5ykj6SFhGZ41Q+ISKzzSvNrFb8nuYZfipwF+CTPNfV6fbk/EYz\nM+ApO9NJERFpLk2VKb7+nhsBWLF8VW3bXntHycFRj3kEAPfft7HW9vOrIxm156L4/3vXxg21trV3\nxuC2ZfPTinP9WfnhjdevB2B92r93wbJa24KFMe/wAQ+NwXj7H5K1taVxfAs6FtW2FdL0wn+7NUo5\n5nVn5Rb9W2I0XXtxNwB6KlnZxZa26E8lDdS77d4Ha23FtpiXeP7CuE6xlB3X0aF5imXWqwDfN7PL\ngG7glUSpw5vcvRLx7IR9lxhQ948p0P4VMU/x6cSMFidMZcdFRGTuUqZYRGablxOrzr0d+AARGL/G\n3T832RO5+zBwHPAtIlv8GeA5wGuA709Rf0VEpAkUKpXKTPdBRERERGRGKVMsIiIiIi1PQbGIiIiI\ntDwFxSIiIiLS8hQUi4iIiEjLU1AsIiIiIi1PQbGIiIiItDwFxSIiIiLS8hQUi4iIiEjLU1AsIiIi\nIi1PQbGIiIiItDwFxSIiIiLS8hQUi4iIiEjLU1AsIiIiIi1PQbGIiIiItDwFxSIiIiLS8jpmugMi\nIrONmS0F3gU8G9gLuB+4AniHu98zgeOfCLwDeDzQC9wEfAH4lLtXpqvf0pp25vVqZuO9Hpe4+8Yp\n6ahIYmZdwNnAm4Br3P3oSRw7bX9fFRSLiOSYWS9wFXAI8CngN8BBxB/vY83sCHd/YIzjjwX+B7gD\nOBPYAJwEfAI4AHj9NHZfWszOvl6T64mgupGtU9RVEQDMzICvAwcDhUkeO61/XxUUi4iM9Hrg4cC/\nuvtnqhvN7I/Ad4kMxb+PcfxngAHgybks3VfN7BLgdWb2ZXf/4/R0XVrQzr5eAda5+8XT10WRYGZL\ngN8BNwOPAW6c5Cmm9e+raopFREZ6GZEdO7du+6XAncApZtYwu2FmjwMMuKjBx9afIrIip0xtd6XF\n7fDrVWQGdAHnA493d5/Mgbvi76uCYhGRxMwWEh9D/87dB/NtqVbtOmAZsN8op3hsuv1Vg7Zfp9vH\nTUFXRabi9Vp/voKZzZvyjook7r7W3f/Z3Qd24PBp//uqoFhEJLNvur1zlPbb0+3+o7SvGu14d98M\nbBzjWJHJ2tnXa9XuZnY+sBnYYmabzOx8M1sxFZ0UmSKr0u20/X1VUCwiklmQbreN0r61br8dOX60\nY0Uma2dfr1WHpttTgJOJWuSXAr8ys913qociU2fa/75qoJ2IiEjrOoEYaPfb3LaLzewO4O3AG4G3\nzUjPRHYxZYpFRDKb0u1odZXz6/bbkeNHO1Zksnb29Yq7/6AuIK6qzmTxtB3sm8hUm/a/rwqKRUQy\nq4EKsHKU9moN582jtN+Wbrc73swWAYvGOFZksnb29TqWdencC3fgWJHpMO1/XxUUi4gk7r4V+BNw\nuJn15NvMrB14InCHu9/e6Hjgl+n2qAZtT063v5iKvors7OvVzB5uZqeb2T4Nmg8iprga7bUusqtN\n+99XBcUiIiOdC/QBp9dtPwVYDnyxusHMDjGz2nRX7v4HYmL6k81sZW6/AvAGYBj4yvR1XVrQDr9e\ngcOAzwLvbHDeah3xd6auqyITNxN/XzXQTkRkpM8CLwE+bGb7EsvmPoxYFezPwIdz+94AODFXbNW/\nAD8DrjGzjxPTBL0QOBZ4h7vfOu2PQFrJzrxevwW8AnhlmmXiCqAdeC5RS/xj4Au74DFIizCzQ8lm\nO6laZmbPy92/wt23MQN/X5UpFhHJcfdh4Djgk8A/AOcBpxIZt6PTH+uxjv818P+I5UvfDXwO2BN4\nhbufPX09l1a0M69Xdy8CJwJvBg4GPgF8hFjw483AM9M+IlPl+cSbseoXRJCc37Z8tIOn++9roVKp\n7Ow5RERERETmNGWKRURERKTlKSgWERERkZanoFhEREREWp6CYhERERFpeQqKZzEzO9PMKmZ23kz3\nRURERKSZKSgGzOx9ZqZpOERERERalILicORMd0BEREREZk7LB8VpecDHzHQ/RERERGTmtPQyz6lW\n99Tc/WoJxTHAmcBTgJOB+en+Q4AD3P0OM1sD7Asc4+5XNTj3qO1m9kxiqcIjgSXAPcC3gf9y9/sn\n2PenA98HSsAz3P2aiRwnIiIiIttr9UzxD4Ev5+7/d/q6M7ft0cA5wHXAl4CBnbmgmb2PCGafBPyU\nWJLzQeANwJ/NbOUEzvEoIoguAC9QQCwiIiKyc1o6U+zuXzezXwKnpfuvr7aZWfXbfwJe5O6X7Oz1\nUnb3DGAtcKS735G2twHnAi8HvgCcMMY59gGuILLXp7n793a2XyIiIiKtrtUzxROxYSoC4uR16fbj\n1YAYwN3LwLuAG4AeM+trdLCZLQb+B9gLeJO7f2WK+iUiIiLS0lo6UzxBP5uKk6QBfUenuz+vb3f3\n24FDxzi+C7gk7fM+d//oVPRLRERERJQpnoj7pug8uxElDwB3TfLYAvAVYuDfT9z9P6aoTyIiIiKC\nguKJ2DJF5+nJfT88yWOfB7wwff9kMztiarokIiIiIqCgeDp11t3fmvt+ySTP1QecD3wA6AK+aWbz\nxz5ERERERCZKQfGOK6fb9voGM+sG9sxvc/cHgAfS3RWTvNb33f1U4D+BXwEHEtPEiYiIiMgUUFCc\nkwbDTdTmdLtng7an0fi5vSrdHtfg2ovMrN/MhtO0a3n3A7h7EXgxMa/xKWZ2av15RERERGTyFBRn\nwS3ECnQT9ed0+3Izq2WLU0D74brzVn0y3Z5uZo/OHVMAziLqjn+RZqJoyN3XAKenu58ys4Mn0WcR\nERERaaDlg2J3Xw+sSXevMbMrzOz0MQ6p+jRRQvE04A9m9gUz+xrwJ+DXxAp49df6GfBfwDzgl2b2\nbTP7HPAH4N+IRT1ePYE+X0isxDcfuDCVa4iIiIjIDmr5oDh5GXA9sAdwOBOYHcLdfwU8E/glsB/w\nEuBRwHuAV4xx3NuBE4FriCnWTgMWA58BHu3uN0+wz68FbkrX/NAEjxERERGRBgqVSmWm+yAiIiIi\nMqOUKRYRERGRlqegWERERERanoJiEREREWl5CopFREREpOUpKBYRERGRlqegWERERERanoJiERER\nEWl5CopFREREpOUpKBYRERGRlqegWERERERaXsdMd2AqffycyysApVKptq2tPeL+SnvcHywO1dqq\n+xWLRQDmzZtXa2tvj6dmcHAgjs8th93eFicrl+P4cqlcayu0FarfxTXK2XFDw8MADAwM1LYNDg6m\n7+IcHZ25H0k6tNq/weHBWtPQUDyOjvau1N+u7a4zXIrjtvX319q2bH0AgCu/+rECIjJVKuvWbZ7p\nPoiMadmyBQDotSpzQXq97tJYpamC4i391V/07QPY4UoEnT19vVlbe7SVUvCYBahQKm0DsoC0um/+\n7NVtxRSEAgwND6W2jpxEP1IAACAASURBVHTbWWurnqsaTEMu2G5LAXq5uF3bcCnOXyxlbW3V4LsQ\n+2zr31JrK7TFtQudESgXitnzUWnLgmcRERERCSqfEJGmZmZHm1nFzM6c6b6IiMjspaBYRKadmf2n\nma2a6X6IiIiMpqnKJwaKUavb2ZmVLNQKB1K5wXCu1KFantDWVi2jyMoayuVU49vRMeI+ZGUQxbT/\niLa0bXAorlMgK8moPzdkJRjD5QYlHMXSiD6XcuUT7YVCeljVthEXSDfx+IpDWR11Z0c7IruSme0H\nvAf4BbBmZnszPU5846Uz3QURkSnzpTOOnekuzAhlikVkuh050x0QEREZT1NlistpMF31FrJBal2d\nPcDIWSSq31dvC4VskGM1g9uorZQysf2D/duds9rWlgZMtufGTQ6n7HE+G9zVFQPfetLMFwNDWcp3\nKH0/nBLEpeHcILzyYLp2XKBAlh3vTAPsKsT1KsUsO14obZ+5FpkuZnYV8JR092dmBrAfsBo4F7ga\neC9wr7s/NtX9vgs4xt2vqjvXjYC5eyG3rQD8M/Aq4BBgALgSeIe73zJO354FfBf4CfB37j481v4i\nItLcmiooFpFZ513AvwInA2cCfwX6UttK4CzgbOCeHTz/J9P5zwc+CqwA3gQ8zcwe6+6rGx1kZo8B\nvgH8FniuAmIRkUx1+r5W01RBcWdHZEjzGdVqhrc4nDKwucxte6qvLZfS1Ge5OYyrM7C11eYkzrLP\n1Uzv8FBcZ8TcwqmOt9IWt9WaZIBiMc5RKmf/fwcGo1/dvfPjum1ZxrejvZL6GdsGC1n/BrdV+5zO\nVchlkdNxbWly5q6O7EFvHVCmWHYdd7/azI5Jd69296tyA+6OA57s7tfuyLnN7JFEQPwVd395bvvv\niWzxGcDpDY7bF7gcuJPIEG+p30dERFpPUwXFIjKn3LOjAXHygnT71brtPyZKNu6qP8DMFgNXEKvl\nPMPd1+3E9UVEmtJsWOBlJrLVCopFZKas2cnjD0u3t+U3unsZuKbB/p3Ad4ja48PdfWevLyIiTaSp\nguJCmksjvypcW1tsbE+3ldwgvOLQ4IhtpVI2YK4/lTXUVo4jd1wqfxhOZRqFtmwSj740YO7BBx8E\nYNvWrdlxxWq/sussWBDvhDoKcf7u9mygXaGzuhpf7F/qyPqwLZV6VAcVdnRmK9UVOqrlE+lqpazs\nortHU7LJrLGzqYjq8pRDY+6VeS1QTT0cB/xxJ68vIiJNRFOyichc0Vt3/750u3iCxw8DJxDB8Nlm\ndsRUdUxEROa+psoUl1I2t5IL9cuFNFit2pbLFPf3b4t9ypGdLbR119qKlfTUpKxwsTSQtaVp3sop\ni9zVm9W9FNOCG8XhtHBGMRvkVl3Qo7sn27bhgRh0358SuFu3bKi1LV26Kto2RT83brq31tbdFX2t\nzhTXkcsid6RscyllwrduyRJy3Z1N9SOX5lMdhdqd32hmvcRsFXlr0u3DiFkt8vu/GNjq7vlVNT7p\n7j8ws9uB3wBfN7PD3X0rO+myj5w0K2rwRMZSrdHUa1WkMWWKRWS6VWuCeiawb3VqtsfUbX8N2/+9\nqga8r8xvNLMjgQuAkxpdwN2vB94MHExM6SYiItJcmWIRmZWqcwW/3cwOJWZ/GM2VxAIcb00LfdwB\nPBE4Bvg18Ljqju5+nZmdC7zSzC4Bvg3sBbwBeICYF7khd/+0mZ0AnGZmV7r7hTv42EREpEk0VVA8\nNBglC+VyNpisOritlEoKKrlBbsViW7qNtvbcILf2NF3wwFCsWjc8nJVPDA3F933z4qOoro7cOQfi\n09/dFkZSrNDWV2srtMVcxPPmZU/7zbdcH21pMNzmDXfU2lat2A+AjrTWwQPr+2ttw+lT5mKaP7lY\nyuY3rvRHfzY/+AAAmx7cWGtbvGCi5ZciU+ZiYvq0pxEzP1w32o7ufpeZHQ+8D3grUU7xU+CpwHkN\nDjmdKJ14JfAFshXtznD328fp1yuAPwGfNbP/dfe/TeIxiYhIk2mqoFhEZh93HwBOrNtcaLRv2v9q\nIjtc72kN9i0BH0tfo53vqkbXc/f7gD1HO05ERFpLUwXFQ/33AyMH03WmFeUGBmLbwFA2XVupFN93\nd0dWty+Xwa2UIivbMy8ysIVcOeTgYGRpOzqjben8rO1hhx8KwH57PySu35VlcNfdH4Plf/3rX9W2\n7b3bonS9yBQv6T2o1tadKii758WYo1Ur96i13bU2Bt1tWB9rD5Rz//Lb2v8/e/cdZ1lV5vv/c86p\nXJ0TdNPQTXwAAUExSxAZRr0qY0CdKyM4zh0xDqZrGkbkemVMoyJixIQ/vQaMI8aRIMIAipJZpM7d\ndKzuylUn/f541tl7U1RVV3VXUd3nfN+vV7921Vp77b129enT6zz1rLX8m54e/3mUhtJnLvZrgoWI\niIjISJpoJyIiIiINr64ixffe9VsAhgbTtfybmz3Kum1nNwCdc+YmdcVhP6+t3Zc/tcMPSeqG+v38\nwSHPT57duSSp6+yYD0CheQ4AR648KKm7/05Pl3zgDr/2ihUrkrrhuPTbl798eVK2dYtHessxmtuU\nT0O+1ZxvyNHa4RuCtHSmn2FaYp/bZ3u+camaRoP7B3oByONlA31pLvJQ30T3ORARERFpHIoUi4iI\niEjD06BYRERERBpeXaVPbNnwIADl4XSJtDXrfLLZogMWAzDUsyWpG447zLW0eJpCaedDSV256OkT\n6x/11IOB/nSZt9mdiwB40cvPAaBaSlMkPvfZ//DzB7zd8mXLkrozXvxCADqXLEzKVm/e6Pcb9N3n\nSvEI6YS5zmZPeSjuTJ9rTtFTNzpmt8RnTifQ7druk/DKcYm6wbhzH0BBn4NEREREHkcjJBERERFp\neHUVKT7+yKMA2L4tjZpu2tQHwAFLfDLc8FBfUnfKc54KwFOe4sf/vvm6pG71mocByLX4RLidcUIc\nwI6tjwBw1+23AXDysYcmdbm4kUeh3Zdie2TT+rR/O7YDcODidNLenXF5t0LOo8JpLBgKOf/M0lrw\nv6aOtpakrjkGrru2eCS8UEg/3zRX2+O1fGJfUyFdoo5K5msRERERARQpFhERERGpr0hxoeRx1pbm\ndDMN8jFi+8jGWJc+8p9vvx+ANWs82rpp06akbmePb+XcM+jXap61KKkbqvq2yQO9vvTZwEAafd7V\n43WDcfm12v0Btm31aPNBBy1Pykrdnntczj/+80lLLHv+6c8D4JgTjknq+no95/m6a28EYNmydDm5\nvv6Yg4xHhdetfTCpKw6n21WLiIiIiFOkWEREREQangbFIiIiItLw6it9Ik5Ma5s1Oy1r9Ulnm9fG\niXL5dGm1det84ltt8llTczqRrRBTMCpNnv7Q1NaR3ijvKQi5Zq877PAjkqoXnOXLrl33R09rePTR\nzUndxnU+6e64Y45NyloXeF/Lw55uUc4syVaser/Cg57+UMpMmOvs9P7kYx8qmRl6nXP8mn1D3s+W\n9tbMc2minYiIiMhIihSLyD7FzK4zs+ruzwQz+4aZVc1sZfx+Zfz+G9PZRxERqT91FSmuRU0H+otJ\n2aCvdEbbAp8oV02rKMfJcDEgS7lYzNT519WSX6BcKqcN42kt7R6RPemkpyVVT33yiQC86S1vAeBH\nV/84qevp2gXAnDlzk7J5S7xfu7rjMnKZpdUqMdJ79333AnDfI+mEuXz8PFMpxyh3031J3cGH+hJx\n8xYu8HMoJXUbM0vESWMwsxcDTSGEn8x0X6bB5cB/Alt2d6KIiMh46mpQLCKjeg+wCqi7QXEI4U/A\nn2a6Hy9510+f0Pt97X1nPKH3ExFpBEqfEKljZpYHnjLT/RAREdnX1VWkuFT2VIed27clZa0lX0uY\nOImu1JquG1zN+de5apy0Vk4/I5TLnnJQKsbciko6Aa6Wb9He3unH5llJ1dxOn6B32hlnAnDtH29K\nmxV8kt/Kw9Id8A5eeiAAfTs9faJj9rz0Pp2estHe4f1rbUvXX66lc+Ti5ML29vakrvYUmzeu8mcY\nTvteGZxQqqbMEDN7GvBe4DRgLrAJuAW4KIQQ4jkXAx8CnhdCuG5E+/sBCyHkzOx84Oux6jwzOw/4\ncAjh4nju0bXrAAuBHcANwCUhhLsy1/wGcB5wKPBm4HXAHOB24E3AfcDFwPnAPOBe4H+P0rdnAB8E\nnh3bbwF+G/u0epSfxRzgE8DZwHzgQeBjIYSrRuvbaNfInLcI+DfgpcAyoBv4I/DREMItY7UTEZHG\noUixyD7CzE4ErgeeDlyKDzKvBM4EbjGzgyd5yWvxQSzAdcA5wPfjvY7DB9tnAV8F3gBcgQ/Gb459\nGenjwDHAB4AvAM8CfghcBjwVH2B/Cjge+KGZJZ/izOz5+ID7JOAzwD8B3459usXMDhrlft8FFuCD\n2YuAduBbZvayyfwQzGw+cDM+mP9efNZPAicCN5iZchFERKS+IsVDwz4Drm/XrqTs4CXzvW7AI6tr\nH92R1A3GyHJTq+9IV8ylUeR8m09S65gVo8C5XFLXu8WjyC1NHnW95847kzo74igATjvVI8VvviC9\nX1eXR7CPO/pJSdkr/sfZAKw/4VEAFs4/IKmrRYZndXofOmelEem2Vl9mLR93vctGkVeveQiAq773\nJX+uoXSpuf7mdPc92eccB9wKfCiEcH2t0Mw2A1/EI6IfmejFQghrzOyX8ds1IYQfZqo/gUdrnx1C\nuDlzr2tiHy4FXjjikouA54cQqvHco4EXAVuBUzLli/DB+LOB38e2lwMV4NQQwqrM/f6MD9T/FY86\nZ20PIbwuc+5P8Kj0B4AfM3EXAYfFZ02iwmZ2FXAP8GngyZO43oxbvHj27k8SGYNePyKjU6RYZB8R\nQvh2COH02oDYzGab2TxgdTxl5VTcx8w68QjxndkBcezDbcDdwJnZSG/0zdrAN7ojHq8ao3xpvN/R\nwNHAb7ID4uhHwC7gxaN09Usj+vYA8FfgZDObzP/qr8YH08HM5tX+AH149PqEGE0WEZEGVleR4nK1\ntjxZ+lirVq8BYMUhSwFYsiCNtm7u8qipHbsQgCrDSd2GDb7C084ej+BSyPwfXI3R1rIfNz66Lqla\ntGAxACsPOgyAfzz3/KRuYMDzhpcvW5aUvfmfLvAvKh6lLpfSzynFokekq3G4sWNHmitdrpQe86x9\nff1J3ca1/szFQY+cD/YPJnU9uxQp3leZWQ6Plv4zYMDIQelU/Xs9Ev9AfPcY9QGPWh+KDyZrVo84\nb3g35bVfvRwdj4+7XwihbGYPAU81s/YQwkCmerT+PYJPHDwEj/KOy8zm4jnEy4CucU49ZDf1+5St\nW3tmuguyH6pFiPX6kf3BTPxGo64GxSL7uUvwNIL7gHcDDwFDwLHA56fwPrVPhmN9QqoNTDtHlA+N\nPHE35Xtyv+ygeLT/uWv17aPUjab2rnoHcOE4562e4PVERKROaVAssg8wsybgX/Bo5akhhG2ZutYx\nGz7eRAaLcUkWZo1RXxsMT1U4aaL36x1R3jFGGUA/E1N7hpaRq2GIiIhk1dWguLbn3NID08lq7S0+\nnjjUDgHgljvuTeqqg/7/bX/Ff9u7ckn6f/YJK32ptN4Br9u6Lf2/+Z57PFg11O//3z7zWU9P6uZ0\n+AS90pC3WzB3QdrB+b7cWmtLOqGvrd37V4q/cO7P7MY3POQBuMFBP27evCmpGxzyMUE1pozcf9/9\nSd0f/uhztNav2eD9HEgDeaXhdHc72acswqOa12UHxNGpI76vvUgeM1g2s3Zg+QTu9QD+z+X4MeqP\nxaO/I/N/91TtH93j7hc/DBwBrAohDI6oPga4bUTZEUCVCUZ2Qwi7zGwDcKSZLQkhPGbnOzNbNMrP\ne9J+/qmz9StpEZH9nCbaiewbtuMD1UNibjEAZnY8cG78thYFrn06OnnENd7K4/9N1z4rJvnJIYR+\n4BfAcWb23OzJZnYans/88xDCMFMgTpC7E/gbMztsRPVr8Q8DV4/S9J9G9O1YfGB9c3yGifo+HgB4\n+4jrzQf+mlmhQ0REGlhdRYpLZf//v7UlXYLs6KOOBGDZcl8GdfX6R5O6jVu2ArDuIV82bdVfNiR1\nC2b5RhsHH+wT5w4/dEVS173Yo8Z33+HzgC655JKk7nmnPR+A5zzdxxrNnWlaZrHiAb6hUvpj7x/y\n4Njmzd6XjRvTaPCO7d6v7p5uALZsSfu+ZYuft2q1B/MeeOCBpG7nTp8vVCl5FLlcLCd1VJB9UAih\naGY/wtft/XYcqB2JL232WnwQ+/y4IcevgUHgvWYGsA5f/ux5+NrDz8hc+lE8D/cFZvZ+4MG4NNt7\n8Aj0T83sMjwqfCTwFmAbvoHIVHorvlHHtWZ2RezXCfH5HgY+OuL8CnComX0HX9ZtQexbjkksSxd9\nBN8A5ANmdgC+FvQBwAXx+E/jtBURkQahSLHIvuNN+IYWZ+IT604BXhZC+A0+Ca8F30CjHXgBvjrD\ne/F1dhcCz2dEDm4IoQi8Eyjh6/U+N5Y/gA+ef4sPTK/EN7X4GfD0EMIjU/lgIYQ/xHvfFfv8FeAV\n8fisEMLIlR968A8I/cD/Bf4PvnTbq0MIk4rshhB2AM/Ef6Z/A3wNn9D4EHBmCOFXe/hYIiJSR3LV\nav1s+/uiFzy3CmkeMcDsdp+Xc9ByXwbtjvvSVZ5uvd13st25wyO4xWI2jOq/OW5u9c8NHW0dSU21\n7D+zwaLn6lYq6c9w2YG+9Nv73/cBAF72qnOTup64qchVX/5cUvbXmAvc1R+Xd1u/JqlbsNCXiivE\n7aF37dyZ1PX1ev5iV5eXnXTSSUldOUbMH37QV9Pq7k7HScUhzykeLpbT3UhEZG9VlVMs+zotySb7\nk/h6fULHKooUi4iIiEjD06BYRERERBpeXU20y+FpDE2FdKzfHJc/6+n3yWptbWlqRWveH39OR2zX\nmk7QK7R43VBcWq2/L10tqhrvU2jx+yxfmi4B19Pt97nmtz8D4JQ48Q5gTrPfe9XttyZl27b4ZLpl\nx58AwH333pnUrV23FoB8vhCfq5DUdbb78nFHHHYUAAcftDKpK5Vjn3v92gsXzkvqBgZHrnolIiIi\nIooUi4iIiEjDq6tIcUecVNeSWZKtVPKJZd27fFLcnNnpBh3PfMZTASgO1yLFaSQ2V/BJd3193m7N\nmvVJ3fr1GwHonOP3mz07jT739Pkkt/CgT+j7P5d8IKk76Wjfu6DQlH4WWXmwLxXXPmsuACsOPiKp\nm3PcHABOfqovR2tHWVK3YL5vCtISo9sPZpZku/7633v7eX7NwcF0Sdd8U/qMIiIiIuIUKRYRERGR\nhqdBsYiIiIg0vLpap/j8c/+uClAuFpOyXFx6OFfw58w3pUvetbb6rrmVuOFbuZK2q+JpF5XYvlRK\nf05dO3294fWP+g54vQN9Sd1wxduVYsOhobR/ne3zvW7H9qTs2aedDsDigzxt4pRnPiepO/wIL1se\nd+Pr2pHub3DPvfcA8Oc/+6S9e+5JJ+h19/h5+WbvQy6zyl9Pj69ZfPstd2udYpGpo3WKZZ+ndYpl\nf6J1ikVEREREZkBdTbSryeczS7I1NcWyuIxac1pXyMcPIDmvq1SynxF8Kbdq1c8pZJZrmz/bJ7DV\ndpr70513JHWVGIkeLBVj+/RH3N034GWZqPOKuKTaUMnvvW3L5qTujr/+BYC1a32XO9+Z161b78u1\n7erZEUtKSd3Cxd6/jtmdALS2tSV1Cg+LiIiIPJ4ixSIiIiLS8OoqUlyMy6/lKmkkthCTiitlL8vl\n01hpDq8rxM0+WlrSpdVyOS/L5/xHVIsY+/letnzpcq/LpZ8t/nKP5/oO9nrOVnlwIKkrDXo+b2tT\nc1L222t+A8BQ2fvyvW1ppLg7bgRSW1Yun92UpDVuPDK3Lfa3ktQNDfsSbKVuT5au7OxO+1BKI8oi\nIiIi4hQpFhEREZGGp0GxiIiIiDS8ukqfqMa0iUo1TSUYLnsKQY5aXZpa0dTsj5+LS7I1P2YSXjzW\nfkK5tF2u6g3mdPhEtiMOWZn2IaZp3P/QgwAUi5l0jZxPzNu+LV2S7aFVPnmuucVvNGtWZ1J35FK/\nbmen75zXmpnsVywPA5CP6SDDw8NJ3VD8un/Qj9sy98vpc5BMgpmdD3wdeH0I4RtTfO0qcH0I4fSp\nvK6IiMieqKtBsYhMuWuBc4DbZrojIiIi06muBsXFzKYdNVUPztLc5F/UosMA+byXVeJGG6Vy5XF1\no21uUoqRYuLku+w5Rx5+GADLDloKQO/AYFLXF5dk27hpY6aDfli4aBEAba3p8mnNzT4hLxd33+gf\n6E/qBgb7H9P31ky7WXHhtcKu7ti/TOfraLMWmX4hhDXAmpnux77uJe/66WO+/9r7zpihnoiIyJ7S\n79JFREREpOHVVaS4HPOHs5t31NTyjbOB0tr5tU048pn9kGvR39oxe81adHaw5Dm7tQ1CvIEfZrV7\nHnAtHxhgR8E32ujanllarcWjwbM62+P90rpahLh2bG5Oc4p7evviM3hf2tvbk7ra0nSdnb6lZ0tL\nWlep7WktDc/Mnga8FzgNmAtsAm4BLgohhHjO+YzIKTaz1fhuMa8GrgSOBpYAC4BVwA+AjwGfBE7G\n/1XcALwzZHegGb1PBrwfOAtYDGwF/gJcHEK4LXPe6Xhqx0XxeClwUqy+EXhLCOGREdc+G3gn8BT8\nvW8V8F3gEyGEQUREpKEpUizSgMzsROB64On4gPJ8fIB7JnCLmR28m0vk4vnfB/4JyA4qVwI/BW4G\n3gRcgQ9yrzWzueP06SB8QPtS4HLgPODTwAnAH83s5FGanQD8OD7LW4H/B7wAuHrEtd8G/AQfoL8n\nnvsX4MPAz8xMmz2KiDS4uooUi8iEHQfcCnwohHB9rdDMNgNfxAekHxmn/aF4RPmjmba1L58GvCaE\n8L1M3QBwMT74/uwY13wScCfw1RDCdzNt7wR+BVyAD8CzXgk8K4RwS/z+m2Z2GHCGmR0WQnjEzA4A\nPg78J/DSEELt90VXmtkm4F3A3+GD6ymxePHsqbqUyJTT61NkdHU2KH5sykP261xu7EBQ7ZxaOsVo\nddn2yc55Me2ikMtM3ovXKBfjpL2mNBg/Ky7hduiKlUlZbfm4oWLswyjz4GoTCIeH0mXXarvv1VI/\naikdkC67lk9270vrRntGaTwhhG8D3659b2azgQKwOhat3M0lcniaxGi6gB+OKLsaHxSfyhiD4hDC\nb4DfZPrUCTQDa8fp0x8zA+Ka24AzgKXAI3jkuQ34HjA3M3gH+BE+KD6dKRwUi4jI/qfOBsUiMhEx\nXeBNwD8Dhg8asyby3rB6jPL7QwgjP33VllxZsZt+vQp4B3A80DmierQ+PTxKWS2Vo7af+rHxeNU4\ntz5kvH5N1tatPVN5OZEpUYsQ6/Up+4OZ+I1GXQ2Ka5POcpmNNmoT5Moxkprd5KJWl8vVorlpu9p5\ntWXRshPthmOkuCmWlTLR1+a4HFol9qWcieAS79PeNispSiK8eY8G92eWcKv1oXb9cmbJuNr+JENF\nPycbHU/Or0XAM5PrRltiThrSJcC/AvcB7wYeAobwAeTnJ9B+KIQwPEZd7yhl3fHYOtYFzewNwFeB\ndXhU+V6gH5/Ad/UYzSYyQa72zvoO4K9jnNM1geuIiEgdq6tBsYjsnpk1Af+CDwRPDSFsy9SNOWid\nhI5RymoT7LaNUlfzbqAMnJldpcJG5DvsgVpYbGMI4bq9vJaIiNQpDYpFGs8iPHp6XXZAHJ06Bdc3\nM8tlJrSBT8yDNI1iNIcC60ZZtm1v+3RPPD4HXy0jYWYtQFsIoftxrSbh5586W7+SFhHZz9XVkmzD\n5RLD5RKDxeHkz8DwEAPDQwwOD/ufoWLyZ6hYjn9KDBVLlMrVzJ8KpXKFwaFhBoeGKZbKyZ98vol8\nvolKJU+lkme4VE7+FHNQzEGlOU+lOc9gqZr82dU3+Lg/27q62dbVTW/fIL19g5QrPO5PJf6pVjN/\nylWq5Srliv8pVUn/VCqUKhWGykWGykVKlVLyp5yrUM5Vdv/DlHq2HY/IHpJdiszMjgfOjd+2j9Zw\nghbhk9uyXhGPfxin3WZgsZklkea4NNzb9rJPP8NTQ841syUj6i4EtpjZVHwYEBGR/ZgixSINJoRQ\nNLMfAecA3zazXwJHAm8GXgv8Anh+3LhjwR7c4i/AZWb2LDxKewI++FzN+JPdvoevIXy1mX0HOAh4\ne2z7aeBEM7sg9m/CQghbzOy9wGeAm8zsMmAXvmnJefjGIjdN5poiIlJ/6mpQ/Mtf3agF+EUm5k14\n9PQs4MXAn4GXhRBuNLNL8MHpx4EP7MG1twCvBz4BvAWfwXoNcGEIoW+cdhfjq2C8HPgCcDdwQQjh\nZ2Y2C98h71J8At6khBA+a2Zr8Ml2H4n3WR2vd2kIoTTZa46Q09qvsr/Qa1VkdDmtRiAiU8HMVuJb\nJ/86hPCCGe6OiIjIpNRVTrGIiIiIyJ7QoFhEREREGp4GxSIiIiLS8JRTLCIiIiINT5FiEREREWl4\nGhSLiIiISMPToFhEREREGp4GxSIiIiLS8DQoFhEREZGGp0GxiIiIiDQ8DYpFREREpOFpUCwiIiIi\nDU+DYhERERFpeBoUi4iIiEjDa5rpDoiI7GvMbAHwIeDvgKXANuAa4KIQwqYJtH82cBHwTKAdeAD4\nCnB5CKE6Xf2WxrQ3r1cz293rcX4IYeeUdFQkMrMW4CPAu4EbQginT6LttL2/alAsIpJhZu3AdcDR\nwOXAn4Aj8TfvM8zsqSGErnHanwH8ElgHXAzsAM4GLgMOBy6cxu5Lg9nb12t0Lz6oHk3fFHVVBAAz\nM+A7wFFAbpJtp/X9VYNiEZHHuhA4HnhLCOGKWqGZ3QH8GI9QvHOc9lcAg8ApmSjdVWb2E+DtZvb1\nEMId09N1aUB7+3oF2BpC+OH0dVHEmdl84HbgQeBk4P5JXmJa31+VUywi8livw6NjV44o/ymwHjjX\nzEaNbpjZMwADha1GgQAAIABJREFUvj/Kr60vx6Mi505td6XB7fHrVWQGtADfAp4ZQgiTafhEvL9q\nUCwiEpnZHPzX0LeHEIaydTFX7VZgMXDoGJd4ejzePErdLfH4jCnoqshUvF5HXi9nZp1T3lGRKISw\nOYTwphDC4B40n/b3Vw2KRURSK+Jx/Rj1a+PxsDHqV47VPoTQA+wcp63IZO3t67VmkZl9C+gBes2s\n28y+ZWYHTUUnRabIynictvdXDYpFRFKz47F/jPq+EeftSfux2opM1t6+XmuOjcdzgXPwXOR/AG42\ns0V71UORqTPt76+aaCciItK4XohPtPtzpuyHZrYO+CDwLuD9M9IzkSeYIsUiIqnueBwrr3LWiPP2\npP1YbUUma29fr4QQfjViQFxTW8nizD3sm8hUm/b3Vw2KRURSq4AqsHyM+loO54Nj1D8Sj49rb2Zz\ngbnjtBWZrL19vY5na7z2nD1oKzIdpv39VYNiEZEohNAH3Ak8xczasnVmVgCeDawLIawdrT1wUzw+\nZ5S6U+Lxxqnoq8jevl7N7Hgze6OZHTJK9ZH4EldjvdZFnmjT/v6qQbGIyGNdCXQAbxxRfi6wBPhq\nrcDMjjazZLmrEMJf8YXpzzGz5ZnzcsA7gCLwzenrujSgPX69AscBXwT+bZTr1vKIfzR1XRWZuJl4\nf9VEOxGRx/oi8Frgk2a2At8290n4rmB3AZ/MnHsfEPC1YmveDFwL3GBmn8GXCXoNcAZwUQjh4Wl/\nAmkke/N6/QHwj8Ab4ioT1wAF4OV4LvHvgK88Ac8gDcLMjiVd7aRmsZm9MvP9NSGEfmbg/VWRYhGR\njBBCETgL+BzwCuAbwHl4xO30+GY9XvtbgFPx7UsvAb4EHAj8YwjhI9PXc2lEe/N6DSGUgJcA7wGO\nAi4DPoVv+PEe4EXxHJGp8ir8w1jtD/ggOVu2ZKzG0/3+mqtWq3t7DRERERGR/ZoixSIiIiLS8DQo\nFhEREZGGp0GxiIiIiDQ8DYpFREREpOFpUCwiIiIiDU+D4r1gZhebWdXMLp9Em9Njm9XT1zMRERER\nmQxt3vHEWw98Ftgx0x0REREREadB8RMshPAQcOFM90NEREREUkqfEBEREZGGp0jxKMzsROB/A88F\nDgAGgQ3AfwKfCSFsHKVNK/BvwKuB5UAfcB3w/hDCA5nzTsf37V4TQliZKV8NrACeDCwCPgg8BWgD\nHsa37LwshFCZwkcVERERERQpfhwzOwX4b+A1wCP4PvJX4z+r9wD/bWaHjGiWB34BnIsPeL8LlIGX\nA9eZ2dxJdOGlwK+BFnwP8F/je9J/GvjyHj2UiIiIiIxLkeLH+yDQClwQQvhSrdDMcvgEubfhOcHv\nzLR5NXA9cEwIoT+evxS4C1gKvAL42gTv/2/Aa0MI38/c+3nA74A3mNmXQwi37uGziYiIiMgoFCl+\nvEPj8aZsYQihig+Ynw38+4g2tUF0f+b8TXj0GODISdz/uuyAOF7rWuA38dtXTeJaIiIiIjIBGhQ/\nXojHz5nZYwazIYSeEMLNIYQtI9rcPkoZQC33eMkk7n/NGOU3xuOJk7iWiIiIiEyA0ice7x3AU4HT\ngAfM7E7gv4BfAdeGEIqjtFkzxrWG4rEwifs/PEZ5bYC9dBLXEhEREZEJUKR4hBDCw8AJwP/FJ9qd\ngA+Ufw1sMLO3jdKsPIVd6BujfDgeW6bwXiIiIiKCBsWjCiFsDyH8awjhcDwf+C3ADcBi4DIzm87N\nN9rGKK+tYLFtGu8tIiIi0pA0KN6NEMJDIYQrQginAW+MxW+dxluOXO6tZnE8bp7Ge4uIiIg0JA2K\nM8xsvpmdY2YvGOOU78TjQdPYjbHu/Zx4vGca7y0iIiLSkDQofqwDgO8B3zKzo0epf3U8/nUa+/AC\nM3tJtsDM/gY4E6jG/omIiIjIFNLqExkhhPvN7FLgA8CdZnYdsAr/8HAMvkZxH/CuaezGx4EfxXs/\ngEelXwTkgM+FEO6cxnuLiIiINCRFikcIIXwQ3575V8CxwOuAf8DXGv4ycFII4aaxr7DXfg88HygB\nfw/8LT44fjvwL9N4XxEREZGGlatWqzPdBwHMbDWwAnheCOG6Ge2MiIiISINRpFhEREREGp4GxSIi\nIiLS8DQoFhEREZGGp0GxiIiIiDQ8TbQTERERkYanSLGIiIiINDwNikVERESk4WlQLCIiIiINT4Ni\nEREREWl4GhSLiIiISMNrmukOTKUbdg5WAYrFUlI20OeraxSL/n12sY0q/k1zs/8Y8oW0slgc9LJ8\nMwCFfEtSN1zy6/cODwEwNFRM6nLxR1oux/sOpX0pFHIANDWlP/ZypeztypV4TlqXz+fjNdMeJ+3K\n5cc8e6FQSM+KDQrxubIPPTjoz/XGpy9NLysie6u6dWvPTPdBZFyLF88GQK9V2R/E1+sTOlZRpFhE\n9mlmdrGZVc3smZM49/QnoGsiIlJH6ipS3B4jq61N6Vi/vdMfsXfYvx8oVZK6gT6P8A4MeNS10JRG\ndXP5WlktWjuc1LW2etkBbc3x5Na0XfyR1oKzA8Npu5qm5jTqXCz6fXr7/LymXPqhqDlGlIdj1Lk0\nnEaHKzGyXIs6Nzc1p9es+nMMFWvHtN1AJnItUoe+D9wN3DPTHRERkf1LXQ2KRaSxhRDuBe59ou/7\nknf99Im+pdSpr73vjJnugkjDUvqEiIiIiDS8uooUF0sx9aCapiCUKjGPocnTDdrb0wlphWZPOSjG\nDIdqNf1xVKueEpHL1ybHpZ8favPXCtXa7L1RPlvk/T6trek1izF1o5pmcJDD+9DR2ebXJk11yMUT\nk2l8mRSJpjjxr1rxc4Yy1yTndbU5e83VNGWiqant8X0VmSFm1gy8FXgdcCj+nrQW+AHw0RDC0Ijz\n/xfwTuAwYDvwLeADIYRKrL8Y+BDwvBDCdbGsCtwGvAa4DHgu0Az8CXhPCOHWaX1IERHZLyhSLCIz\n6TLgP4D7gXcAbwJuAi4Cvjvi3DcAFwBXxPMeBd4by3dnPvAr4GHg7cBHgZOA35nZoXv9FCIist+r\nq0jxn+5aDUBLaxopbm3pAKAal1sbHOpP6jo6OgGYNWsWAAMD6dJqpRjVLZdrZWm0ta/Pr9HS7FHX\nHIVMnS95Njxcin1J62qrqG1YtyUpqy0V19zm0d3Zmcl+xcEB/6Jzkfe9nD5Xbbm2tjaPaLe0pJP3\nqjFSXi4VaxdK6pprz/HUJyOyD/ifwD0hhL/PlF1lZg8BTzezzkz5M4Cn1aLHZvYrYB0eAf7Kbu5z\nBPC+EMLHagVmth74Bj5IfsfePojIVKgtm7a/30Nkf6RIsYjMpBJwkJmtzBaGEP49hPDyEEJfpviL\n2XSKEMJGPIVi2QTvNXLg/KN4PHVyXRYRkXpUV5Hin/3mFgCamtME2/Z2jwLPafeA02BfV1K35ICF\n/kXcVGPHjl1JXbnkYd1SOSYc59Jc3+GSR50rOY9CZ3OKu3f1Apkl0spp9LkvrgtXHE7PHxj0+krV\no88rF7UndU854SgA/vKXvwJw37ptSV1Tm0epW1s9UjxnztykLleJz/PoJr/fzk1pXf8OAC5+5RcR\n2Qd8GPgscL+Z/RL4HfDrEMJDo5w7Wlk/0D5K+UibQgg7sgUhhB4z6wFWTLLPItNmOjfW0OYdsj+Z\nid9oKFIsIjMmhHAZcBY+GH4BcDnwoJndaGbHjzj98Yt+T1zvGOXdQOsYdSIi0kA0KBaRGRVC+G0I\n4cXAAuCFwFXAs4Hfm9m8KbpNxxjlc4FtY9SJiEgDqav0iXyv/3a0kE8npO3c7v/fDXZ4GP6A2Wk6\nw8r5SwC4fZVPfNvQlaZP5Cv+eaEcd5MbGkzb5Ws70hU85aE0nJ0c5ymPc2Z72kaBalJXzfk1m9oz\nS6sVvG2p4ukQ/eW07vBDlwPQHhdl63l0baYPft3WJj/u3JhO3uva6s/cs/VRf/Zt65K6uU3pc4js\nS0IIA/gKEb8ys2345LfTpujyS81sbggh+UduZouBWcCde3vxn3/qbP1KWkRkP6dIsYjMCDN7qpk9\nENceHqk7HodGqdsTeeD8EWUvj8c/TNE9RERkP1ZXkeL5Cz0a3JxPx/qtcVmy+fM86tq/5cGk7qF7\nVgFQaF8KQHt7+hvWHLUlzuImHO1pxDdX8Ovn4oYg5eFM9LXsZS1N3q6cSyf95ePGIYPFTFlta464\nWUjvzoGk7ld/8L6uWDwn3nd+UhfnAdJU7YzfpxMBafH7zF7qkfB82wFpu2J6fZEZdgcwAHzezJ6M\nb6ZRAp4MvA24B7gWeOYU3GsNcIGZHR7vsxL430AXnscsIiINTpFiEZkRIYQSvhzaZ/DJdpcDVwJn\nA58HTh25o91eGMTzlZfjq128F7gVOCOEsH6K7iEiIvuxXLVa3f1Z+4mzLr27Cun2yACFuKlFe9Wj\npy0DG5O6yrBHVyuzPFI83JQGznNxmbVCXFqtkM/UFTy629zum2K0ZNqVY35xadj7sCMz531g0HOd\nS9ltnuPyccUYbe7vTqPOleJOAFqbPYpcKqT5xpV4y2pcfq2UpjVTKvuzFgrerpyJIpfiiauvODlN\nvBapY3Gb5xBCOHoab1NVTrHs67Qkm+xP4uv1CR2rKFIsIiIiIg1Pg2IRERERaXh1NdHuwAW+DFo+\nsyRbPi6DVmrysmp1TlpXjJPh2vxXSq25dG+AlqqnGTQ3e7tMhgS5eM3mJr9Wa3Oa1pCPqRHliqel\ntPWmKZHdPX7NoVKazlCu+j2HBjviNdNrDZd8o67apL/5HelnmKa4tFo+/hUWS2kaTGXIr1lIrpP2\nvZJrQUREREQeq64GxSIiI4UQlD8vIiK7VVeD4o4lHi2tZpZBa6r415VynHRW6kzqKjn/vzJXW96s\nOf2/sz1Obmtujsuv5dNIbD7vode2Nr92a3NmsmKMAhfjJL420r4UY6A3l0/LSrG+uc0n7bXl0ihy\nsTwU++yR33K1kNTVrlDIxWduSqPItRXpmgq5x54MVCuZpdtEREREBFBOsYiIiIhInUWK8zFHOBMa\nndXqObotccONSiWNtubjlsqVuIQZLemPIx+XP4uXJJeJ4OabPFI8Z1a8b3N7UlcueuS2f8jPae/I\n5DfHKPLQYH9S1tbmF+kr+f36h1uTuuYWv0Yl3xr7mV6rWhmOz+Nlte2oAUq1Jelqp2eiw5Vimjct\nIiIiIk6RYhERERFpeBoUi4iIiEjDq6v0CYY9haClkElZiEuxNbV4XaGpLamr7Vo3HHeTI5e2y8UU\njFztY0Nm57/huNxacWAAgN6hdBm1QtHb9W3b7PcfSLe0O3pevFRL2odVG3cBsKnPUzC2VNIl49qH\nfEe7npJ3omkoTf0o4ZPwBoux75mPN83Nfl5tZbrhgXRZuPJgTJ/4+5WIiIiIiFOkWEREREQaXl1F\nittmebS1rS19rHLZI6lDpbh8Wi6dFFcqx4lscbJaU2YyXSEum5bPedQ1l4kiN5X860LFI769s9II\n7qxVtwGw6N47AFgzPDupu7vTI8qtcxclZes2bfX+zfKyyvyVSd1wzpdp62j1Z+hsmZf2ocO/LlW9\nrqk1naBXiEvENcdl2rIT9CqZCXkiIiIi4hQpFhEREZGGV1eR4nLZI6t9fem+xgODnvdbiOuTDTal\n2xxXakuXVb2uLbN5R+0n01SNS7q1pO2SL2vbNffuSuo67/8jACvCTQAsnrMyqVu/1ZdfW3BgGile\n1uJ92BajwQMrLalrW/Y0AOYUYt5wJf3r6o95xrXn6+4fTOoG47Jr7R2+dXQ5s81zT08fIiIiIvJY\nihSLyJjM7Hwzq5rZ+dNw7aqZXTfV1xUREdkTdRUpFpEpdy1wDnDbTHdERERkOtXVoLi2alp2Klkt\n7aGQ91SHttbMhLlmn5xWjUusNeXSH0euUjvP6waHiul9ZnuAfcFCT604ZEd6zTl9ns7QFH+0/f2Z\nVI6d2wBYNjud0Ne7ayMAG7d1eX8rafC+v3UBADvj5LhSqSftA3FyX9x5b1fPQNouftmzphuArq40\nvaM4FO/9yiMQ2Z0QwhpgzUz3Y1/3knf9dKa7MG2+9r4zZroLIiJPCKVPiIiIiEjDq6tIcaFQi/xm\nxvpxabWmGFgdHk4jt8ND/nVrXM6st5jWleMkuqGhuPFFJvy8oNk32GiLhaU1jyR1le3rAZg77wAA\ntvcMJ3WrujxSPGddGvFtXXYYALs2+iYfLWtCUlc8YgcA3ZVO/560f035uFRcwf8KewbSv8od2/1a\nO3f4sa8vnYTX0Z5uDiKNzcyeBrwXOA2YC2wCbgEuCiGEeM75wNeB14cQvhHLVgMl4NXAlcDRwBJg\nAbAK+AHwMeCTwMn4v54bgHeGEB7YTZ8MeD9wFrAY2Ar8Bbg4hHBb5rzT8dSOi+LxUuCkWH0j8JYQ\nQvoP09ucDbwTeAr+3rcK+C7wiRDCICIi0tDqalAsIhNjZicC1wPb8AHlo8ARwIXAWWZ2fAhh3TiX\nyOED4u8Da4HsoHIl8FPgW/Gc4/DB6LVmdmwIYRejMLOD8AFtAR9QrwYOAt4O/NHMnh1C+NOIZifE\n+i/Fez0X+CfgatJBMmb2NuCy+MzvAYrAGcCHgVPM7G9DCFrEexSLF8/e/UmyX9Hfqcjo6mpQvGWr\nb4tcKafR2ULc8rmj1Zcn27p5W1K3s8vzeA897HAAci2ZJdmSzTs8Z7ecS/+/7N7uUeTN2/xaWx64\nNak7sftRAI6f58uuzZuVbrHcnPOtn9d3p9s8H3His/1adz8MwILMsmvDJR9n9JRiRDoTrh7s9aXV\n+vr9mYfL6QYifd1x2bWYi1wcTnOYS01pbrQ0tOOAW4EPhRCurxWa2Wbgi8B5wEfGaX8oHlH+aKZt\n7cunAa8JIXwvUzcAXAycD3x2jGs+CbgT+GoI4buZtncCvwIuwAe8Wa8EnhVCuCV+/00zOww4w8wO\nCyE8YmYHAB8H/hN4aWbwe6WZbQLeBfwd8ONxnldEROpcXQ2KRWRiQgjfBr5d+97MZuMR2tWxaOVu\nLpHD0yRG0wX8cETZ1fig+FTGGBSHEH4D/CbTp06gGY9Ej9WnP2YGxDW34VHgpcAjwEuBNuB7wNzM\n4B3gR/ig+HQ0KB7V1q09uz9J9gu1CLH+TmV/MBO/0dCgWKQBmVkOeBPwz4Dhg8asibw3rB6j/P4Q\nQnlE2cZ4XLGbfr0KeAdwPNA5gT49PEpZLZWjOR6Pjcerxrn1IeP1S0RE6l9dDYqLFU95KBfT9ImW\nqk+664vLmeUzc/DmzpvnZTlPPegZSFMLWnKeSrF4kU9M29nfm9Rt7/L0hPYm36Fu6ZzFSV1Ts0+w\nayl5X3bmZiV1KxYvAeC+jVuSsoX3/9mPA9v92jvTNIht27w/+Vn+f/vQtu6kritOois0e10us1Nf\nDr/3nNk+zjlw0aJMXfqzkYZ2CfCvwH3Au4GHgCF8APn5CbQfCiGM9WLqHaWs9uJtHeuCZvYG4KvA\nOjyqfC/Qj0/gu3qMZhOZIFcLN7wD+OsY53RN4DoiIlLH6mpQLCK7Z2ZNwL/gA8FTQwjbMnVjDlon\noWOUsrnxuG2Uupp3A2XgzOwqFTYi32EP1H5XvDGEcN1eXktEROpUXQ2KDz7QN7toys1LyoqDvpNF\nz4AHtZpb098St7S0AzAYI8RdO9M8q+Eu/7ow6O3aMkOFti6P6h4T408H5rYndfNa/bfGc+Mkublz\nDkjqjp17MADL29Loccds7wPLlwGwqpz+lezc5MGrnl2rAdixNo0wD1Q8ojx7to8/hgbTgFlzzqPG\nu5o9LN7W1J8+14BP9uPdpyANaxEePb0uOyCOTp2C65uZ5Uas5nBoPG4crUHmnHWjLNu2t326Jx6f\ng6+WkTCzFqAthND9uFaT8PNPna08TRGR/Zw27xBpPNvxiOwhMbcYADM7Hjg3ftu+F9dfhE9uy3pF\nPP5hnHabgcVmlkSazexg4G172aef4akh55rZkhF1FwJbzGwqPgyIiMh+rK4ixSKyeyGEopn9CDgH\n+LaZ/RI4Engz8FrgF8Dz48YdC/bgFn8BLjOzZ+FR2hPwwedqxp/s9j18DeGrzew7pGsUXwh8GjjR\nzC6I/ZuwEMIWM3sv8BngJjO7DNiFb1pyHr6xyE2TuaaIiNSfuhoUP7za1whuiWsMAzTlYyCsySek\nDQynk+lyfb5DXF+fp1is37A5qevZ4CkRD99+BwBHLk7TLs590nIAlj7gK0HlV92c1M2OaROFxX7O\nrNKOpK7wiO9296S2NL1juM/TM3KL/bfLmwaak7r77v5vADZv975Ui+lkusGYZtG1yu9XGU7TJ1pb\nPaA2u90n71cH0l/rlkuaaCeArzwxhO8c92Lgz8DLQgg3mtkl+OD048AH9uDaW4DXA58A3oLvaHcN\ncGEIoW+cdhfjq2C8HPgCcDdwQQjhZ2Y2C9/Q41J8At6khBA+a2Zr8Ml2H4n3WR2vd2kIoTROcxER\naQC5arV+NnF6zRfuqMIkBsVxxabaoHjdKINiej2vd9RB8frHD4prucTNcVBcKaT5vIW1PiiuZAfF\ns70PG2f5oPiGgXQli9/s8r+b8QbFwwN7Nih+NHwls1OJyN4zs5X41sm/DiG8YIa780SrKqdY9nVa\np1j2J/H1+oSOVeoqUvzIev+HXuzP/IMv+yB4QVxaraU5jcQOD3lwqFj0yXG1Xe8A5h7ibx49G/zv\no3fDHUndHb0+b6c4x3ers0XpgLkQ7925MEaAC+n9hrp8UJtrSpddK+3cCsCKNp9o9+JjDk7qNq/b\nBMDGZT5ZrykzqX/do77q1XDRn6t729ak7ugn+TWWxHZ33nx/Urf2/nTgLyIiIiJOE+1EREREpOHV\nVaS4edijupVM2mxrk0dxd23yFISh4aGkbu58T1WoxL23Omalk9s7WnwNti19nj6x9o7/SurWdvuK\nUbOX+UT2g1rSlIzZFY8UHxPTLlYsOyapO/opT/cv+tLfBrSsXwXAnFaPKM+al6799rxDTva+H+Cp\nGDs2p6tn5e/dAEA38/1ZutIVpZrneUR5a8nvU5y7Mn1mSyPRIiIiIuLqalAsIjMnhLCaJzj/S0RE\nZKoofUJEREREGl5dRYo33PlnAJpb0seqln0yXSWuQtE0O91Nrr/ZJ6tVKl43MJiuFLE2piP09MXU\niDnLk7o5PWsAOHDdOgDaMjvUlWIqxgNdPqHtz03rkrpypy/52jZvaVJ2YFwVY8Uiz/nobJ+T1PUc\n6ekT5L0vByxO2y0/1FM9NvZ6ukV1YZoW0d/vfR6IO/UdfHzarpRZpUJEREREnCLFIiIiItLw6ipS\nPBzXCB4cSCfTkfNx/7wDDwKguS1d67djVnz8nEdri+V0qbRK0dsNDntEtrr0zKSua8jvU912IwAH\n5NO1npe1ervTnuzrDrc3p3Xbt+8EYMfgqqRsx06/1j0Pe19y23uTuuOOPQ2A+UsO9ILWdCLg/IXe\n1/aF/jxdPWkEOL+z9uwetu7u2p7U9ezaiYiIiIg8liLFIiIiItLw6ipSfPCRhwAw1J/uJDtrlu/q\n1j7bd5HrG+jO1Hm0tbnNlzBbuCjNy80Pe929d68G4MGNaXR3e8zZXe8b4XE36Q6xnU3+OePAp58K\nwLEHWFLXet9fADhq84NJWVuvL7O2Y4fnFOePOzGp2zDsfb9ng0e+Z89LJ/ZXYgS8VtLRmUaR5y3w\nyHep5M/Q05N+9unp7UREREREHkuRYhERERFpeBoUi4iIiEjDq6v0iWNO8MltlXI5KRsc9Alou3Z5\n2sTC2WmaQVObfyaYO282AG3tabtiyc8/7HCv62RJUrd+0FMQhucc7dfpSHeh6xnwiXKzlnnd4qcf\nl9Td0L0FgG3DaRpE8/yjACgs9zSKXaTb8XVtetTvvcB35du6c0f6XPEZWzu8fwPFNIWjJe/P1ZTz\n9ImBoXQCYf9gByIiIiLyWIoUi8g+wcy+YWZVM1u5m/Mujuedvpf3m5LriIhIfairSPG8+R6x7exo\nS8qGhnw2XHjAlyJrbk2jpvMX+kYZ+YKXPbp1Q1KXL3u7JnzJswOWNid1x7/Cl0pr7YjXqqaR31tu\n+CMAhx+8AoCnHJVOnJvTMheAzVu3JWUD3R5ZXnXr7QD0bU2XVuvt8rpys58/b868pG77Vo9k53J+\nzsBQugxdoVrxZ49L0zUV0uhwMTntZET2U98H7gbumemOiIhI/airQbGI1L8Qwr3AvTPdDxERqS91\nNSieP8ejucOD6QYYs9r9EQ9autDrqulmGtu2bQVg48aHANg1kF6rEIO/TXHps8H1dyd1fff/GoCO\nGICd3Tk7qdsRo8D3/Mm3nF7YkkZp737Y73PLrX9OytY+4GXdax4B4JBjnpnUVQ893Ps84H3oLacd\nHO71ZeH6e3q8T709SV37HI+Y9/ft8ms/mm7eMdRd28r65YjI1HjJu346010Y09fed8ZMd0FEZL9Q\nV4NiEakLLWb278BrgSXAOuALIYRPgecCAx8CnhdCuC6WVYH/Aj4BfA6YE0I4MNYtBj4F/A+gHU+7\n+PAT+DwiIrIf0KBYRPY1nwZagH/H36POAz5pZpUQwqfHadcBXAFcBmwGMLM8cA2eRP9N4HrgoHje\nQ9P1ACIisv+pq0FxruppBhs2pekC1TjprDdORCtV0vSJrtpEtqKXdbTMSuryFU/FaC57msLwQLrk\n2eZV6wGolLoAaMqnk/BycSLf1f/5YwBuvuPmpO7uu+8EYPu6zUlZR8nzNBa0+TXuueuupG5o1wEA\nzDnU/5rmdLakfd/qy7U1F7xu59YtSV3fNr/m3PlLvaA3ndg3tGUnIvu4TuCMEEIFwMyuAlYB7zOz\nz47T7pnAP4QQ/r9M2YvxAfFVIYTza4Vm9j3gLhrA4sWzd3+SNBS9JkRGpyXZRGRf85XagBgghLAD\n+B2eSnHsOO3KwE9GlD0/Hr+bLQwhPIinW4iIiAB1Fin+/fW3ALBtZzrprFryzTDmzvUIbmkwnazW\n2+XLnw12o3EjAAAgAElEQVT2V2JdunlHueQ/mqEen6w2uP6/k7p8uc+PxAhxpZjUlQb9680bPHJb\nTS/J0E5v115II8vtTR79rcayvmL6V1Lu9mXXeh7d6NeeMzep69vskeJc3KAjV04j2f1dXte3wTf7\naCmknWhpSc8T2UfdPUrZI/G4Ypx2W0IIfSPKDovHB0c5/z7gRZPs235n69ae3Z8kDaEWIdZrQvYH\nM/EbDUWKRWRfM9r/2LVPs+2j1I3Xrrb8S/8odQOjlImISIOqq0jxI488DEB7W7p5R3veo6Qdgx4x\nLhbTzTGKec8lnn+gL9fW15/8xpZc3GhjsNuv1V89OKnrqnguMdVSPKRbMze3+P/ZB63wAFXfzq1J\nXb4St19uSSPFtXzkwZjrPJCJOld6PNLb1+N9zrdmtqiu+P/xxSE/v7k53UCkWvYIc7nk/+cP5dNI\ncT6vSLHs80bbi3y8we14agPftlHqZo1SJiIiDUqRYhHZ1xwzStkR8fjIKHXjWROPh41Sd9wkryUi\nInWsriLFIlIX3mBmPwwhVCFZZ/j5wAYgTPJa1wNvBs7BJ+sRr2nAaVPTXfj5p85WnqaIyH6urgbF\nTXj6w8HLFidlzzrpaACOO8bTH+59IA00PbTeJ8O1zvb0icHBNLVgsN+/bmv2iXBzTzspqVt3/18A\n2LD6fq9rLiR1Ha3+W94lBxwEwMN33pbUbX3Ug1a9vbuSskLe25bxXehoS1MdahkRuYr3pTycpkBW\nc55S0RRTQEqD6fyifNGXn6vEdpVcZrZfWekTss+rAr8ws58DrcAb8FSHd4cQqj6enbAf4xPq/peZ\n5YCb8XWK34gPkl84lR0XEZH9l9InRGRfcz6+69wHgY/hA+O3hhC+NNkLhRCKwFnAD/Bo8RXAy4C3\nAr+Yov6KiEgdyFWr1d2fJSIiIiJSxxQpFhEREZGGp0GxiIiIiDQ8DYpFREREpOFpUCwiIiIiDU+D\nYhERERFpeBoUi4iIiEjD06BYRERERBqeBsUiIiIi0vA0KBYRERGRhqdBsYiIiIg0PA2KRURERKTh\naVAsIiIiIg1Pg2IRERERaXgaFIuIiIhIw9OgWEREREQaXtNMd0BEZF9jZguADwF/BywFtgHXABeF\nEDZNoP2zgYuAZwLtwAPAV4DLQwjV6eq3NKa9eb2a2e5ej/NDCDunpKMikZm1AB8B3g3cEEI4fRJt\np+39VYNiEZEMM2sHrgOOBi4H/gQcib95n2FmTw0hdI3T/gzgl8A64GJgB3A2cBlwOHDhNHZfGsze\nvl6je/FB9Wj6pqirIgCYmQHfAY4CcpNsO63vrxoUi4g81oXA8cBbQghX1ArN7A7gx3iE4p3jtL8C\nGAROyUTprjKznwBvN7OvhxDumJ6uSwPa29crwNYQwg+nr4sizszmA7cDDwInA/dP8hLT+v6qnGIR\nkcd6HR4du3JE+U+B9cC5ZjZqdMPMngEY8P1Rfm19OR4VOXdquysNbo9fryIzoAX4FvDMEEKYTMMn\n4v1Vg2IRkcjM5uC/hr49hDCUrYu5arcCi4FDx7jE0+Px5lHqbonHZ0xBV0Wm4vU68no5M+uc8o6K\nRCGEzSGEN4UQBveg+bS/v2pQLCKSWhGP68eoXxuPh41Rv3Ks9iGEHmDnOG1FJmtvX681i8zsW0AP\n0Gtm3Wb2LTM7aCo6KTJFVsbjtL2/alAsIpKaHY/9Y9T3jThvT9qP1VZksvb29VpzbDyeC5yD5yL/\nA3CzmS3aqx6KTJ1pf3/VRDsREZHG9UJ8ot2fM2U/NLN1wAeBdwHvn5GeiTzBFCkWEUl1x+NYeZWz\nRpy3J+3HaisyWXv7eiWE8KsRA+Ka2koWZ+5h30Sm2rS/v2pQLCKSWgVUgeVj1NdyOB8co/6ReHxc\nezObC8wdp63IZO3t63U8W+O15+xBW5HpMO3vrxoUi4hEIYQ+4E7gKWbWlq0zswLwbGBdCGHtaO2B\nm+LxOaPUnRKPN05FX0X29vVqZseb2RvN7JBRqo/El7ga67Uu8kSb9vdXDYpFRB7rSqADeOOI8nOB\nJcBXawVmdrSZJctdhRD+ii9Mf46ZLc+clwPeARSBb05f16UB7fHrFTgO+CLwb6Nct5ZH/KOp66rI\nxM3E+6sm2omIPNYXgdcCnzSzFfi2uU/CdwW7C/hk5tz7gICvFVvzZuBa4AYz+wy+TNBrgDOAi0II\nD0/7E0gj2ZvX6w+AfwTeEFeZuAYoAC/Hc4l/B3zlCXgGaRBmdizpaic1i83slZnvrwkh9DMD76+K\nFIuIZIQQisBZwOeAVwDfAM7DI26nxzfr8drfApyKb196CfAl4EDgH0MIH5m+nksj2pvXawihBLwE\neA9wFHAZ8Cl8w4/3AC+K54hMlVfhH8Zqf8AHydmyJWM1nu7311y1Wt3ba4iIiIiI7NcUKRYRERGR\nhqdBsYiIiIg0PA2KRURERKThaVAsIiIiIg1Pg+J9mJldbGZVM/vGTPdFREREpJ5pUAyY2aVmpmU4\nRERERBqUBsXuaTPdARERERGZOQ0/KI7bA5480/0QERERkZnT0Ns8x1zd8zLf11IongdcDJwGnAPM\nit8vAw4PIawzs9XACuB5IYTrRrn2mPVm9iJ8q8KnAfOBTcDVwEdDCNsm2Pe/AX4BlIG/DSHcMJF2\nIiIiIvJ4jR4p/g3w9cz3n41/1mfKTgK+ANwKfA0Y3Jsbmtml+GD2ucDv8S05dwHvAO4ys+UTuMaJ\n+CA6B7xaA2IRERGRvdPQkeIQwnfM7Cbg9fH7C2t1Zlb78gLg70MIP9nb+8Xo7vuAzcDTQgjrYnke\nuBI4H/gK8MJxrnEIcA0evX59COFne9svERERkUbX6JHiidgxFQPi6O3x+JnagBgghFABPgTcB7SZ\nWcdojc1sHvBLYCnw7hDCN6eoXyIiIiINraEjxRN07VRcJE7oOz1++4eR9SGEtcCx47RvAX4Sz7k0\nhPAfU9EvEREREVGkeCK2TNF1FuIpDwAbJtk2B3wTn/j3XyGED0xRn0REREQEDYononeKrtOW+bo4\nybavBF4Tvz7FzJ46NV0SEREREdCgeDo1j/i+L/P1/EleqwP4FvAxoAX4f2Y2a/wmIiIiIjJRGhTv\nuUo8FkZWmFkrcGC2LITQBXTFbw+a5L1+EUI4D/hX4GbgCHyZOBERERGZAhoUZ8TJcBPVE48HjlJ3\nJqP/bK+Lx7NGufdcMxsws2Jcdi1rG0AIoQT8T3xd43PN7LyR1xERERGRydOgOB3cgu9AN1F3xeP5\nZpZEi+OA9pMjrlvzuXh8o5mdlGmTAz6M5x3fGFeiGFUIYTXwxvjt5WZ21CT6LCIiIiKjaPhBcQhh\nO7A6fnuDmV1jZm8cp0nN5/EUijOBv5rZV8zs28CdwC34Dngj73Ut8FGgE7jJzK42sy8BfwX+Bd/U\n458n0Ofv4TvxzeL/b+/Ow+Q6qzuPf6uq9127LMlavB0h40AcBxtPwBg7DpAYksyYZAIDxDAxBJJh\njQPEsQPOMkkISVgCYQhgEpiAgyEJBobFslkcg80gY2y/sq1ua5e6JXW3eu+uqvxx3rr3qtWttSW1\nqn6f5/FT0n3vvfVWq9x96vR5zwv/HMs1REREROQE1XxQHL0KeBRYBlzKMXSHCCHcD7wE+B6wDngF\n8GzgvcCNR7ju3cD1wH14i7XfBLqADwM/HUJ44hjn/DvA5vicf3GM14iIiIjIDHLlcvlMz0FERERE\n5IxSplhEREREap6CYhERERGpeQqKRURERKTmKSgWERERkZqnoFhEREREap6CYhERERGpeQqKRURE\nRKTmKSgWERERkZqnoFhEREREap6CYhERERGpeQqKRURERKTm1Z3pCcylnwz0lQFGR0aSY5OTkwBM\nTU0BUCqWkrFS2f/c2dkJQFtbWzKWz8XPC6V6vw/F9Lqc/7mQy/ljeksK5TIAubyP5Qp1mev8WLFU\nTi+I55dLmZtU5pAvHDLPEuXDzilVri+nY7ly5bEc75PLXOF/tvaO7EEROTnl3t6DZ3oOIgAsWdIO\ngN6TMl+c6HtyyZL20xqrVFVQXJ6cAKA4MZ4cGzk4BECh4AHmwYPpP0hzS7OfMzgIQHv8O0BDgwfD\nQxN+z519+5Kxji4Pojua/PzGTOBbyHnwnYuRaa6QScbHYDUb3BZjsr4cA9dDgtt4rFTyx4liGpgn\nz5ecnD16aPA8Q0wsUjPM7AXAPcAfhRBuO7OzERGR+UrlEyJyypnZH5jZ2jM9DxERkdlUVaZYROYf\nM1sHvBf4DtBzZmdzalz/ti+d6SmIiMw7//D7LzzTUzgu1RUUx1KHXKwjBsjHkoN9vXsBKJXTEoQN\nF54HQKHOvwz5TN68qc7rDLbt3w/AHZ/5v8lYS8dCAK541rMBuOT885KxxYtaAKiL19eX01rhfCyN\nKGWOTcR6hmLlWClbGxzrkuN1dZmyiEqVRbbcYvpgZSRXytRM5FQ/Iafdz57pCYiIiBxNdQXFIjKv\nmNlG4Kr413vMDGAd0A18HLgX+GNgdwjhOWZ2G3ArcHUIYeO0ez0OWAghlzmWA94AvA5YD4wBXwNu\nCSE8eZS5vRS4C/gm8IshhMkjnS8iItWtqoLi1oYmAAb37U+ONcRM7EBfn/+9IVmaxlC/n9fY0AhA\nMbOQbbzR7zUVs87D4+nYY4/5z9rHNvcAcMmac5OxFzzvUgBWLF0EwHkrVyRj9TETXZ9J1paIGeJK\npjifzQZXukfk4/XpP1cxdqtI5pxtaJHcP2aaj7AIT+QUuxV4I3ADcBvwE6Aljq0C/gi4Hdh1gvf/\nQLz/HcBfASuBtwPXmtlzQgjdM11kZpcBnwUeAn5VAbGIyNyrdJ2Y7e/zTVUFxSIyv4QQ7jWzq+Nf\n7w0hbMwsuLsOeF4I4bsncm8zexYeEH8qhPCazPH/j2eLfx+4aYbr1gD/DmzHM8RDJ/L8IiJSXaoq\nKC6UYv3wrt3JsYlKnXHsU5xJtvLAd74NwOjoKAANDQ3J2Lmr1wBwYMJbs3UtWZ6Mtazb4M+zdw8A\nPQP9ydiX7/F7rljYBcCNv/7yZKy11Vu4FTJ1zY0F/3NdzAaXM/XGxdhTeXIqtprL9Equ9DDOxzRw\nLn94rXBSSpzNIitTLPPHrhMNiKNfi4+fnnb8G3jJxo7pF5hZF3A3UAJ+IYTQexLPLyIiR1DpS3wS\nfYrnfE5HUlVBsYicVXpO8vpnxsct2YMhhBJw3wzn1wNfwGuPLw0hnOzzi4hIFVFQLCJnyslut1XZ\nbWfiGM//HaCSdrgO2HSSzy8iIlWkqoLi5thabUF7ul1zZQe7vrj18+TYVDL2xOYAwM5dvsanqbEx\nGTsQF+Zt3uXlhv3t5yRjq5/hi+naVq70e+7bmYy1jY8BUFfn5Q0jE5ntm9vj/Yvpjnt5fD5TRf+5\nvnv3nszcfae9A/0DAAyPpGuBzlvnbeDOPbeyyC8tn8gVKm3eVCohVaV52t/3xscuZiiVmMEk8GLg\nz4DbzexbIYSH5nB+IiJyFquqoFhEznqVT36N2YNm1ox3q8jqiY8X410tsuf/BjAcQsjuqvGBEMJX\nzWwr8CDwGTO7NIQwfLKT/rf3vey4a+VETpUTrd8UqXVVFRR3tHunJ7voguRYd7d3ZHoiPAbAvgN7\nk7GB2JLtYP8BH5tMs8j7+z1DnFuwFoDCwqZkbHDYf4a2L/bFdKWJtBC8IbaF6+rsBKA/k5kej0nj\nxkwCd6roMcD27VsB+MKd/5KMbdu6DYCR0fH4mGaKn/vcKwG44eU3ALAgLuwDyFV2787FTTzKh7d5\nEzmNKitEm454lqu0ZrsM7yBR8SYO35b+S8C7gNcCn6scNLOfBf4J+EQ85xAhhEfN7B3AB/GWbjce\nw7xERKTKVVVQLCLzUqVX8LvNbAPe/WE2X8M34Lg5bvSxDbgSuBp4ALi8cmII4ftm9nHgtWb2ReBf\ngHOAtwAH8L7IMwohfMjMXgz8ppl9LYTwzyf42kREpEpMz7yIiMy1O/G+wJcB7wQWz3ZiCGEH8CLg\nEeBm4P3AIuAaYKZ+wjcBbwUuAD6GZ47vA34mhLD1KPO6EdgDfCT2LhYRkRqWq6Zfp+/t21IGmJpK\n+/n29PQA8IW77gJg/8BAMjY27D9jS/H8/YNpaeF4vhWAJRc/H4Dc4nShXePiBQAsWu4/23c+/VQy\nNtnrC/TWrF4LwCXrL0zGrtrgZR1tU+nP9qd3+GK/7373OwB8+V+/nM7voPdPzpX9s8voRPq6Fi/1\n5/6fN70OgGuufWEyli/4+fn4mSe7U1/lz4tbFh7e2FhETlRZ9ZsyX6imWOabk+hTfFpjFWWKRURE\nRKTmVVVN8c6d3hpteDjN+G7Z4lncp5/uAaA/kw1uaYoL3GO2PF+fdnzqWuwL3Ru6PCNbLKRfqrFe\n3wRr74F9AAz1pov3RmIrt1JcFNfWWJ+MXXzOIgD27NuWHLv7q74O6MknngRgfDRtuToWF9iNDceF\nduPpWC7uTNdb2b0vkw2ur6vsduefeQqkpqroNwMiIiIic0WZYhERERGpeVWVKW5v95qVkbhRh/Ny\nlJYWb9fW15fWFO/a3w/ARKwpXrYmrf9dsmI1AFOt3kVqcjLdhGNfj2d6N296GIDiSJp97mj1WuTC\nmGd193Wk2ef/KPl5o71ppvihH/wQgLEx3/Qjl0s/pxTqPMtcLPtYKVMrXZ/3f7rJMc8i98daZoBz\nVq7w88s+52qqGxcRERE5FZQpFhEREZGap6BYRERERGpeVZVPVFqxrViR7gbb2uolFQ8+6GUK3ZM7\nkrGJcT9/KpYstHQtScYKrb4jXbHg5Re5kbFkbNdj3katf7Mv4mvKp0vZ+nK+6K4R38luW240HXvK\nyxg6mtLPIk1xB7z6QoNf15Xubjs54eUP7Z1eDrI/UyJRLPnYo4/47rZrVq1MxhYv9AV9pdiaLVs8\noVIKERERkcMpUywiIiIiNa+qMsWbNv0YgHNXnZscG40L2EZjq7PB/nTjjMlizJo2e3a23NiWjJVi\ne7bSpLdW69++PRkb2OoL5ZomfKy+kC7CGy96hrjSmm315RuSsYufeWE8P83Wjg2bP476PJtjZtvn\nHp+73xcH9g8cSMb69uwBYNsOz3zv3JnNgPviu1xsOVcqpfPL5bRnh4iIiMh0yhSLiIiISM2rqkzx\nzp2ePW1pTjO+IXj9b1+vb7RBOfM5ICZQJ0t+bLyU1gaPTMVTRnxLwu2PP56Mje3fD0BDzjO+xVg/\n7Lf0mw4f9HPOXbYwGbvqyssB2LdvX3Ls8Z94dvvghGewi+VMO7m8t2Rr6+oCYPGqdKvpc9etAWDv\nDs9adyxYkIwdGPTMcme91xZns8PKFIuIiIgcTpliEREREal5CopFREREpOZVVflER1sHALt27UmO\ndXf3ADA54SUOLU0tydhw3HWuPrZfyzWkY+OxImJ0dy8AvfE+AKW4kK3Y6F++8Xy6cK4U/zhV9Hvv\n3Z0u0DvQuyce602ObfzWRh8b8JKHhpa09GP1uvP9nnVe1lForU/Gli70soxV69YCELqfTMYGhvxe\nF9kzAFi5Mm3XtnTpUkTmMzPbCFwVQjhqrY+ZfRJ4NbAuhNBjZmuBbuBTIYTXnMJpiohIlamqoFhE\nDmdmvwTUhRC+eKbncgp8EPh3YO+ZnoiIiJzdqioo3rl9JwBPPPVUcmzXrl0ADA/7QrbhwYPJ2Die\neT1nqW/2Ud+UZmmnxjwb3NfTA8BQ3/5krD5u1lFpdFYqF5OxfDxaF9uudWcyuHv37AagOJW2SKtk\nlvMFn8vAwGAyNhzbyTV0eJu2XNzoA2Aybjiy76C/nm9/+7vJWF3Mr61a8QAAV1xxRTL289deC8Di\n9nSjEql678Czp1UXFIcQHgQePNPzEBGRs59qikWqmJnlgUvP9Dyq3fVv+9KZnoKIiJykqsoUi5zt\nzOxngZuBq4BOYBfwAHBLiP0Fzew24Fbg6hDCxmnXPw5YCCFnZq8BPhGHXm1mrwb+KIRwWzx3feU+\nwCJgP3Af8J4Qwo8z9/wksW4X+G3gVUAH8EPgDcBjwG3Aa4Au4FHg92aY2+XAu4Er4/V7ga/HOfXM\n8LXoAP4CeBmwAHgC+N8hhE/PNLeZ7pE5bzHwh8BLgRXAIPBd4E9CCA/Mdp2IiNSOqgqKN37zXgD6\n9qd9gCl4Mnx0xPv/jmXKExoW+c53LYt8IVpdZqHdSL/fY9/T3QCU4852fs/mQ563rpiOtTV6PURr\ni5dD7NyR7jS3Y5eXTyxatCw5dmBw2B/39fs84852AMtXjcT5eb/hunzmn6uyti/uVjcWF/8BDPb7\nvQYP+uP4eHrPurx/PezGdKc9mR/M7NnAvUAf8KfAbuAC4M3AdWZ2SQhh23Hc8h48iP0wsBH4EB6w\nYmbPxIPCKeDvgM140PtG4H4z+7kQwo+m3e/PgWbgXcDFcV53At+M194KrAZ+D7jTzFaFEMbi810D\n3I0Hwn8NbAU2AG8CXmJml4YQdhz6dHwWGMGD2S7gJuAOMxsKIdx1rF8EM1sA3A8sAT4KPAKsxAP6\n+8zsxSGEbx3r/UREpDpVVVAscpZ7JvB94NYQwr2Vg2a2B/gInhG9/VhvFkJ42sy+Ev/6dAjhzszw\nX+DZ2itDCPdnnuvuOIc/BV487ZaLgWtCCOV47nrgJUAv8LzM8cV4MH4lUAk2P4iX4T8/hNCdeb6H\ngM8Bf4AHqVn7Qgivypz7RTwr/S7gmINi4BbgvPhak6ywmX0a+AnwfuBZx3G/GS1Z0n70k0ROI70n\nZb6Z7+/JqgqKt3Z7+7NippFTY6tndScmPbU6WkpfcteiFQDUt3hLtnIxXTDXt/1pAPr3ehu1ukKm\n/Dr+sRyztIVcet3S+A/escCfd+e+tP3ak0/1ALBsxYXJsVLJz9ux9Ql/HtL2buMDnkVumPL710+m\nGd/muPteXVOjP29HukiwMOlZ47ZOb1GXXbz3ja9/A4DX3/gmZH4JIfwj8I+Vv5tZO1AAeuKhtXPx\nPGbWClwHPJwNiOMcfmBmjwDXmllTJdMbfaoS+Eab8KD40zMcBzgnPt96YD3wr9mAOPoCMAD8EocH\nxR+dNrfNZvYj4DIzaw8hHOTY/BoeTAcz68ocH8bLRa43swUhhAPHeD8REalCVRUUi5zNzCyHB4a/\nBRjQNO2Uufr/9UL8o90js4wHPGu9Dg8mK3qmnTdxlOOVxtrr4+NhzxdCKJrZk8DPmFlzCGE0MzzT\n/LbgCwdX41neIzKzTryGeAVwpKB39VHGj6q391hjdJFTq5KN03tS5osTfU+e7sxyVQXFE+OeuS3X\npVnd0ojX+46Oe7Z1rC7NqDYsOAeAyZKfP9bXl4zt2uKt1MbH/Gd0Q74h80x+rzJxQ5CW9Mu4YJHH\nMW0dfv6+wXQum5/sAeCctWnNc2usa57Mx9ZtpYlkrK/Pf0bXP+1Z6/rGNBnX1+gZ4qZ6f57J0fS6\nhtiurdIzrlxOrxs+OIzMW+/BywgeA94OPAmM47W3H5rD56n8TzDbm6ESmLZOOz4+/cSjHD+R58sG\nxTN996yMN88wNpPKd9RNeA30bHqO8X4iIlKlqiooFjlbmVkd8L/wbOXzQwh9mbHG47jVsQSLQ/Gx\nbZbxSjA8V2mmY32+oWnHW2Y5Br4A71hUXkPD9G4YIiIiWepTLDI/LMazmpuyAXH0/Gl/r7Q7OSRY\nNrNmYNUxPNdm/Ncdl8wyvgHP/k6v/z1Rj8bHw54vfhi4AOieVr8M8IwZ7nUB3nul51ieOIQwAOwA\nLjSzw/Y4j4sCRUREqitTXCx6mUCxlC58mxrzsoK4zo762N4MYCLvpQ4H+n0h2nBv2hFqcI/vjper\n3Cuf7kJXLPlviwt1fqyjqyMZa2rzBXBNMV+3eHFnMrZjt+9Eu/GBTcmxgaJ/LsktXg5AeSJNgO0Z\n99hnPLZry1SFUIqfZ5rjLnf9g2lbuKkR/y11+aC/ro62NEG3YmnaDk7mlX14oLrazHKZTg6XAK+M\n51SywLvi42XA1zL3eBOHf9Ct/M+Q1CeHEEbM7MvAS2Prte9UxszsKrye+c4QwgRzIC6Qexj4eTM7\nL4SwJTP8CvzDwEdnuPR1wA8yc9uAB9b3hxCONVMM3t3iLcDv4uUplfstAH5kZj8OIUzvtHFc/u19\nL1P9pojIWa6qgmKRs1UIYdLMvgDcAPxjbKV2Id7a7BXAl4Fr4oYcXwPGgJvNDGAb3v7sanyjj8sz\nt96N1+G+yMzeCTwRW7O9A89Af8nM/hbPCl+I9ynuwzcQmUtvwjfquMfMPhzn9VPx9T0F/Mm080vA\nOjP7DN7WbWGcW47jaEsX3Y5vAPIuM1uG94JeBrw+Pr7uRF6QiIhUl6oKivOxbdpkZjONYskXw+Xr\nfEFa58I0UzxV8vTxwYEBAPp37UzGyqOeba3Lx+xzeSrzTH6srsGfr6m5kBmJzxfbwnW0pw0Enu7z\n3w6HrdvTOS/1tnCNK3zBXWEqXWdU1+r3bTlniY9l/rkmpvwJCvF58oPpGqbJIV+gV5rwY2P5dKHd\n/um/mJf55A142cJ1eIuyh4BfCSF8x8zegweyf44HwC/CewnfjJdTfAu4Bvhk9oYx2H4rHhjeAvw9\nngXeHHeYew8emC7E+w3/K77D3FyVTlTm8W0z+zl857ub8frincDHgPfO0A7tIP4B4X3AH+ObdwTg\n7SGEr3AcQgj7zewKfBOQ6/F+z8PAfwCvy/aEFhGR2pXLdiY4261Zdl4ZYDwTFE8WYzBbCYrPS8sa\nm5etAyAfd4rrf/rxZKy/23e5rYtfn3I5u5ucH2tq8aB41bkLkqHly/28zjZ/HBlPf5v9424Piifb\nz0+OVYLiyRiEZ4Pi5hgULz1CUNwYg+IDW9LuVMO7vZNFacrLJ9pa0p36utp8Mf7Gex/KdHMWkZNU\nVvmEzBdqySbzzUm0ZDutsUpVZYpzcRONumz8GlO2uTp/rC+kHwIaYgZ1OG6HPNi7JxnLT3k5ZS4G\nzDFqCQkAAA83SURBVKVMnXKl41lzswfa+UJabzw07IvlC5XSzkKaKW6IWz8PZoL2/KQH7W3tvqdA\nHWl9cn2rn19u86B7opS+NyYm4nPGIHrp8nS9UNui+BrLXnaZ3ZRkaEDfJEVERESmU/cJEREREal5\nCopFREREpOZVVflEcyxPmCItkWhv9GPtXV6CMNWYfg7IF73Gd/iA7zA3MZyWFrTipQqleK9cZrFa\nU7O3h+1a4DUyLa3pbnelopddTMWyiK7OdFOw87u8hWxXbmVyrG8kPs+4n19oSO9VLvrzDMfurSMj\nA8nYyIiXTbQWvURiWXPaPWtFm99jKM5hbDTdcKy+K60vFhERERGnTLGIiIiI1LyqyhSvWukbU4xM\npQvZKjte5Bo9ezpZSrOmxVHvzjC+zzPFCzNdGjas9RZp9S3+Jdq2c3cytmyZP09ji2dyJybSnWhX\nrPBOEc1N3jmifUmaFc6P+BwmJtPFd4UFvrBub89WAPq7e5Kx5uW+oUfDAs8wt2U2EJmKGem2mLUu\nT/YnY3sP+N4OfZOeRS6Op9d1Nh3PjsEiIiIitUGZYhERERGpeQqKRURERKTmVVX5BFPej7c4mS46\nm4wtesdHvcRhZCrt9dtY8DKLFry8oCnT4HjVKi9deMYlFwBwz733JWOr164G4Nw1awH4wQ++n4w1\n5L08IZ9rBqC7e38ytq3P57B3JF20t+xZlwHQ1R57GPfvTV9Ps88nN7nQ51dKd9XrbPYSjIZ6/1wz\nuD8tC+FA3MmuIe6uN5W+rlKmskREREREnDLFIiIiIlLzqipTPBXblBUyLdlKsZVac1xw15RpyVZX\n9kxqc5tndwf2DyZj27btAKAzLoQbPphuvzw57lnZ5ct9Ud3VV1+VjO3f5Yvcdu3uBWDPjnSB3tYe\nv+fIZJqtrmvwtm7tXW0ANBTSsXx8PQe3+CK84czYePw8s2bVOQB0dC5MxibGfGe+Cy7yBYG7t/Yl\nY22FtOWbiIiIiDhlikVERESk5lVVpnjVUs+aLly2JDnW0Oq1veWYMa6vS9uTNdR3ArBtq7czeyyk\nNbujo16XvOmHPwGgubEtGVu6eCkAI0O+2ceGZ6xPxr69y9u7bXnSs7v796Q1wg1TnmFesiCd38Du\nnQDkOtb5PJvTTO5FF3g98zOvuByAPUNpJrt/zLPIXXHDkvHudGys5K+1q81f+4FMhnnfnl5ERERE\n5FDKFIuIiIhIzVNQLCJzwsw+aWZlM1t+puciIiJyvKqqfGLJggX+uCgtT2hubwVgbNzLDabGB5Kx\n8WEvORgbOgCAXXReMnZgwEsqHn3kUR9bf1EyNjLk93o67j63ds2a9LoD3natb++BeO/hZOycRb4Y\n7gVXPT85tvHhJ/z8UZ9XviPdVa+h3f/c0eGlG30j6c55fbt80V7fmL+GtuG09dvEaGzPNuHlIG2N\naUnGwKR6somIiIhMp0yxiIiIiNS8qsoU9/b5IrfhsbHkWGOjt1sbH/djAwfShW979nk2d/+gZ1Z/\n+jnPTcaaKtfFBW19vekCtYkpPzYy4VngoaG0XVvPU75wbnzC71ksFZOxUtkX+Y2MjiTHFi71lmzd\nu/y6saE0kxu2bAZgsBTvNZUuEmyNm3YsXebZ8aGetO3akiW+EPC8lWv9cXmaAR9amy4KFBERERFX\nVUGxiMwLOTN7J/BbwEpgB/CBEMJfVU4ws5XArcCLgOXAQeAB4E9DCN/OnHdbPO8a4LXA9cA7Qwgf\nMrN24K3Ay4HVQAnYAtwB/E0IoZS5z2LgD4GXAiuAQeC7wJ+EEB44BV8DERE5y1RVUDwS64YnptJs\na0OD19NWmpKV82nNblOrty5rKXprte6nnkzGxsa9JVsxtlHbtXN7MrZ7r3/ZJopes9u9ZUcylsv7\nM9XXxVZpU2mmeDz+iB7IZIr393mGuC22UVvasTi9V9yjem+vn9NIIX2xDf48U2V/fResXJoMrVns\n9c8LWroAaG5uTMYKabm1yKnyHuB84M/i328G3mdmPw4hfD0uxPs+sAD4CLAJD4xvAr5lZteHEL46\n7Z5vwYPe1wMPx2OfA34B+Ds8oC4Avwj8FXAuHjBjZguA+4ElwEeBR/Bg/Q3AfWb24hDCt+b0KyAi\nImedqgqKRWReWAVcW8nUmtkmPCj9deDreOZ3BfAbIYTPVi4ys88CAXg/MD0oPg94dghhMp67EM8y\nfzmE8MbMeZ8ws/cDC80sF0IoA7fE66/MZoXN7NPAT+LzPetkX/SSJe0newuROaX3pMw38/09qYV2\nIjLXPpAtXSDN7K6Ij78CHMAzvYkQwlbgG8B6Mzt/2j2/VAmIoyKeOV5vZoum3ectIYTXxIAY4NeA\nx4BgZl2V/4Bh4D7gp2I2WUREalhVZYq7FvvPxuJUujNdIcb9uZw/5jOlBA3tC+N1XiIxOpoumJuY\n8J+/He1NABwcStuhDQ77or3ChP/cn8i0OZsc98V3oyPeKq2US0se9uz3hX0PP/pYcqx/wBfIHaxU\nVNSlLdxG8fKJwSZ/PYVyMkRD2efQ2+FlGlMrlqXX7fad+ra3eSu3js60ZKSzoxWRU+zJ7F9CCCNm\nBtAcg9FlwPdCCMUZrg3ALwEXAU9ljndPu+eAmf0NXlbRbWb/DnwT+GoIIalnMrNOPBhfgQfis1l9\nlPGj6u09eDKXi8yZSjZO70mZL070PXm6M8tVFRSLyLwwcYSxyn7pw7OMVz6ZTv/0NtN30rfhZRm/\nDdwA/HegbGZfAV4fQtgGVL6jbgLefIR59RxhTEREakBVBcWFOs/KlnPpsXzu0ExxgTTdWsj5iY34\nYrX21qZkLA6Ry/v5lXZqAKOj/jN/YtIzuKVSes+xmCkeGPDNOIZGxpOxkbipRv/u3em9xvxYedKf\n8GApPX8qTqIQk9vNjekL64xz7azzuY+NpdnxvXHh4OCwZ7B396UJsFzmayNyBlR+5dI2y3glGD5q\nOiGWR3we+LyZdeAdKn4T71Dx/8zsksx9GkIIG0900iIiUv1UUywip00IoR/YBTzDzAoznLIhPj42\nw9iR7jsYQrgrhPBS4C5gPXBxCGEAbwl3oZktnX5dbNUmIiKioFhETrvPA114N4qEmV0IXA08GBfd\nzcrMftHMeszsuhmGB+Nj5dcun8N/K/a70+6xAPhRLLcQEZEaV1XlE/WVnsSZhXa5WC+Qz3v8n33B\n5Vj1kM/70UKhLjNWWQNUOuRcgIb25njz5Oz0+WIJY+7clQBMTKRricZi2cVwpqRicMyPjU/EkoyJ\ndNFeqeTP3VDvCbWOtob0tdb7XCt9mOvr69O5lypz9gkWM30ASqWZ1jaJnFbvBX4Z+JiZ/RTeFu1c\nvG/wFPCmY7jH9/D/ne80s7/Dew/ngecCrwK+EUJ4PJ57O/Ay4F1mtgy4F1/s9/r4+Lo5el0iInIW\nU6ZYRE6rEEIfcAXwT8ArgI/ji+D+A3jusewwF0I4AFwe7/Fy4O/xjUB+DrgN37mucu7++HwfAn4e\n+AfgD/AuGdfOsFGIiIjUoFw5mwIVEREREalByhSLiIiISM1TUCwiIiIiNU9BsYiIiIjUPAXFIiIi\nIlLzFBSLiIiISM1TUCwiIiIiNU9BsYiIiIjUPAXFIiIiIlLzFBSLiIiISM1TUCwiIiIiNU9BsYiI\niIjUPAXFIiIiIlLzFBSLiIiISM1TUCwiIiIiNa/uTE9ARGS+MbOFwK3ALwPnAH3A3cAtIYRdx3D9\nlcAtwBVAM7AZ+BjwwRBC+VTNW6rbybwvzexo77sFIYT+OZmo1BQzawBuB94O3BdCeMFxXDuvvlcq\nKBYRyTCzZmAjsB74IPAgcCH+Df+FZvYzIYQDR7j+hcBXgG3AbcB+4GXA3wLnA28+hdOXKnWy78vo\nUTyonsnwHE1VaoiZGfAZ4CIgd5zXzrvvlQqKRUQO9WbgEuCNIYQPVw6a2SbgLjyr8dYjXP9hYAx4\nXiZ792kz+yLwu2b2iRDCplMzdaliJ/u+BOgNIdx56qYotcTMFgA/BJ4ALgMeP85bzLvvlaopFhE5\n1KvwrNnHpx3/ErAdeKWZzZgRMbPLAQM+N8Ovsz+IZ1JeObfTlRpxwu9LkVOkAbgDuCKEEI7nwvn6\nvVJBsYhIZGYd+K+nfxhCGM+Oxfq27wNLgHWz3OI58fH+GcYeiI+Xz8FUpYbMwfty+v1yZtY65xOV\nmhJC2BNCeEMIYewELp+X3ysVFIuIpNbEx+2zjG+Nj+fNMr52tutDCAeB/iNcKzKbk31fViw2szuA\ng8CQmQ2a2R1mtnIuJilyHNbGx3n1vVJBsYhIqj0+jswyPjztvBO5frZrRWZzsu/Lig3x8ZXADXgt\n8v8A7jezxSc1Q5HjMy+/V2qhnYiISPV7Mb7Q7qHMsTvNbBvwbuBtwDvPyMxE5gllikVEUoPxcbZ6\ny7Zp553I9bNdKzKbk31fEkL46rSAuKLSyeLaE5ybyImYl98rFRSLiKS6gTKwapbxSm3nE7OMb4mP\nh11vZp1A5xGuFZnNyb4vj6Q33rvjBK4VOVHz8nulgmIRkSiEMAw8DFxqZk3ZMTMrAFcC20IIW2e6\nHvhefPwvM4w9Lz5+Zy7mKrXjZN+XZnaJmd1kZqtnGL4Qb38123ta5FSYl98rFRSLiBzq40ALcNO0\n468ElgL/p3LAzNabWdIGK4TwI7yZ/Q1mtipzXg54CzAJfOrUTV2q2Am/L4FnAh8B/nCG+1bqiL8w\nd1MVOdTZ8r1SC+1ERA71EeAVwF+a2Rp8O92L8d3Cfgz8Zebcx4CA95Ct+G3gHuA+M/trvLXQrwMv\nBG4JITx1yl+BVKOTeV9+HrgReG3sMnE3UAB+Fa8l/gbwsdPwGqSKmNkG0o4mFUvM7L9l/n53CGGE\ns+R7pTLFIiIZIYRJ4DrgA8B/BT4JvBrPxL0gfoM/0vUPAM/Htzx9D/BRYDlwYwjh9lM3c6lmJ/O+\nDCFMAdcD7wAuAv4WeB++4cc7gJfEc0SOx8vxD1yV/8CD5OyxpbNdPB+/V+bK5fKZeF4RERERkXlD\nmWIRERERqXkKikVERESk5ikoFhEREZGap6BYRERERGqegmIRERERqXkKikVERESk5ikoFhEREZGa\np6BYRERERGqegmIRERERqXkKikVERESk5ikoFhEREZGap6BYRERERGqegmIRERERqXkKikVERESk\n5ikoFhEREZGap6BYRERERGqegmIRERERqXn/CdwrSspi1hQVAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f5fc85b5940>"
      ]
     },
     "metadata": {
      "image/png": {
       "height": 319,
       "width": 354
      },
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\"\n",
    "DON'T MODIFY ANYTHING IN THIS CELL\n",
    "\"\"\"\n",
    "%matplotlib inline\n",
    "%config InlineBackend.figure_format = 'retina'\n",
    "\n",
    "import tensorflow as tf\n",
    "import pickle\n",
    "import helper\n",
    "import random\n",
    "\n",
    "# Set batch size if not already set\n",
    "try:\n",
    "    if batch_size:\n",
    "        pass\n",
    "except NameError:\n",
    "    batch_size = 64\n",
    "\n",
    "save_model_path = './image_classification'\n",
    "n_samples = 4\n",
    "top_n_predictions = 3\n",
    "\n",
    "def test_model():\n",
    "    \"\"\"\n",
    "    Test the saved model against the test dataset\n",
    "    \"\"\"\n",
    "\n",
    "    test_features, test_labels = pickle.load(open('preprocess_test.p', mode='rb'))\n",
    "    loaded_graph = tf.Graph()\n",
    "\n",
    "    with tf.Session(graph=loaded_graph) as sess:\n",
    "        # Load model\n",
    "        loader = tf.train.import_meta_graph(save_model_path + '.meta')\n",
    "        loader.restore(sess, save_model_path)\n",
    "\n",
    "        # Get Tensors from loaded model\n",
    "        loaded_x = loaded_graph.get_tensor_by_name('x:0')\n",
    "        loaded_y = loaded_graph.get_tensor_by_name('y:0')\n",
    "        loaded_keep_prob = loaded_graph.get_tensor_by_name('keep_prob:0')\n",
    "        loaded_logits = loaded_graph.get_tensor_by_name('logits:0')\n",
    "        loaded_acc = loaded_graph.get_tensor_by_name('accuracy:0')\n",
    "        \n",
    "        # Get accuracy in batches for memory limitations\n",
    "        test_batch_acc_total = 0\n",
    "        test_batch_count = 0\n",
    "        \n",
    "        for test_feature_batch, test_label_batch in helper.batch_features_labels(test_features, test_labels, batch_size):\n",
    "            test_batch_acc_total += sess.run(\n",
    "                loaded_acc,\n",
    "                feed_dict={loaded_x: test_feature_batch, loaded_y: test_label_batch, loaded_keep_prob: 1.0})\n",
    "            test_batch_count += 1\n",
    "\n",
    "        print('Testing Accuracy: {}\\n'.format(test_batch_acc_total/test_batch_count))\n",
    "\n",
    "        # Print Random Samples\n",
    "        random_test_features, random_test_labels = tuple(zip(*random.sample(list(zip(test_features, test_labels)), n_samples)))\n",
    "        random_test_predictions = sess.run(\n",
    "            tf.nn.top_k(tf.nn.softmax(loaded_logits), top_n_predictions),\n",
    "            feed_dict={loaded_x: random_test_features, loaded_y: random_test_labels, loaded_keep_prob: 1.0})\n",
    "        helper.display_image_predictions(random_test_features, random_test_labels, random_test_predictions)\n",
    "\n",
    "\n",
    "test_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "PqOmHDOpyGVW"
   },
   "source": [
    "## 为何准确率只有50-80%？\n",
    "\n",
    "你可能想问，为何准确率不能更高了？首先，对于简单的 CNN 网络来说，50% 已经不低了。纯粹猜测的准确率为10%。但是，你可能注意到有人的准确率[远远超过 80%](http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html#43494641522d3130)。这是因为我们还没有介绍所有的神经网络知识。我们还需要掌握一些其他技巧。\n",
    "\n",
    "## 提交项目\n",
    "\n",
    "提交项目时，确保先运行所有单元，然后再保存记事本。将 notebook 文件另存为“dlnd_image_classification.ipynb”，再在目录 \"File\" -> \"Download as\" 另存为 HTML 格式。请在提交的项目中包含 “helper.py” 和 “problem_unittests.py” 文件。\n"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "default_view": {},
   "name": "dlnd_image_classification.ipynb（副本）",
   "provenance": [],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
